Subscribe: Issues for Feeds
http://drupal.org/project/issues/rss/feeds?order=last_comment_timestamp&sort=asc&categories=All
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
case  content  core  cron  drupal  error  feed  feeds  field  file  import  node  user  vagrant web  web core  web 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Issues for Feeds

Issues for Feeds





 



PHP 5.6: Fatal error during import: FormattableMarkup::__toString() must not throw an exception

Sun, 11 Mar 2018 10:20:39 +0000

Drupal version: 8.5.0
Feeds version: 8.x-3.x-dev (2018-03-11)
PHP version: 7.0.22

Hi, first of all: thanks for your hard work on the feeds module!

Problem:
AJAX error (server error 500) + fatal error in webserver error log during first feed import after feed creation. Vanilla Drupal 8.5.0, trying to import into Article CT.

Steps to recreate:
- install vanilla D8.5.0
- install feeds 8.x-3.x-dev via composer
- create new feed type 'blog item', leave all options default ('download', 'rss/atom', 'node', 'article')
- add new feed, url: https://dri.es/rss.xml (tried various other feeds too, with same result)

--> batch job starts, error message is shown:

An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: /batch?id=6&op=do_nojs&op=do
StatusText: Internal Server Error
ResponseText: 

The following gets written to the webserver's error log:

[error] 31600#31600: *128 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Method Drupal\Component\Render\FormattableMarkup::__toString() must not throw an exception, caught Drupal\feeds\Exception\ValidationException:  in /path/to/vhost/www/modules/contrib/feeds/src/Feeds/Processor/EntityProcessorBase.php on line 469

Thanks.




file mapper need to pass language in file_field_widget_uri data array

Wed, 04 Apr 2018 16:02:29 +0000

When importing translatables image fields which are using tokens in their field definitions, we need the file mapper to pass the current mapping language value to the $data array passed to file_field_widget_uri as third argument.
This way we let the opportunity to deal with the key used and translate it in another token name if needed.

Step to reproduce:

  1. install a drupal with entity translation and at least 2 language (fr & en for example)
  2. create a content-type with an image file marked translatable with a file directory value comporting a token: for example 'photos/[vd:language]'
  3. create an importer with at least two mapping of that field (one mapping per language).

Expected result:

The media files should be placed in their respective language folder.

Actual Result:

When launching the importation process we don't have any information to get the correct file directory to save these images.




Add Feeds target for Telephone

Fri, 20 Apr 2018 13:49:14 +0000

Drupal 8 core's "Telephone" module should also be supported as feeds target. Let's create a patch against the latest 8.x-3.x in this issue to get this included.




When mapping to File or Image fields, empty source for target_id causes Exception and breaks import process.

Tue, 26 Jul 2016 06:48:14 +0000

For example, I have a CSV file where not all the rows have image URL.. I am mapping image URL field to Image->target_id.
Empty target_id eventually will throw TargetValidationException.

Should we add a check for empty target_id value in IMAGE class inside prepareValue() ? And if it's empty just skip any further processing of the field ?




Language support in Feed import

Mon, 21 Nov 2016 12:40:28 +0000

I have a multi language site where i need to import content from RSS. But unfortunately i need to import it into different language and i cant find out a way? I do not see language field in target mapping page as well.

Any idea or how to fix it ?

/takim




Update documentation for Feeds D8

Sun, 06 Mar 2016 14:04:57 +0000

Because the UI is not very self explanatory especially when it comes to mapping custom sources to node fields. Thanks.




Field validation errors in item

Wed, 25 Oct 2017 21:20:24 +0000

Upgraded from Feeds 7.x-2.0-beta3 to Feeds 7.x-2.0-beta4 and started receiving the folllowing error message:

Field validation errors in item 'xxxxxxx':

Where 'xxxxxxx' is the equivalent value of the User table Name field column.

Trying to import/update User table records. I receive the above error on nearly half the imported records. Some records still import OK.

PHP 5.6
Drupal 7.56
MS SQL Server 10.50.6220.0 [SP3] Standard Edition (64-bit)




Fatal error when triggering Feeds via cron

Wed, 19 Oct 2016 18:44:49 +0000

When attempting to run an RSS Feed import on cron, I am getting a fatal error. This appears to be due to the fact that the Feeds $entity->original returns null Fatal error: Call to a member function hasTranslation() on null in /vagrant/web/core/lib/Drupal/Core/Field/Plugin/Field/FieldType/ChangedItem.php on line 47 Call Stack: 0.0016 246608 1. {main}() /vagrant/web/index.php:0 0.0196 544288 2. Drupal\Core\DrupalKernel->handle() /vagrant/web/index.php:19 0.0554 1781704 3. Stack\StackedHttpKernel->handle() /vagrant/web/core/lib/Drupal/Core/DrupalKernel.php:652 0.0554 1781816 4. Drupal\Core\StackMiddleware\NegotiationMiddleware->handle() /vagrant/web/vendor/stack/builder/src/Stack/StackedHttpKernel.php:23 0.0555 1782208 5. Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle() /vagrant/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php:50 0.0555 1782640 6. Drupal\page_cache\StackMiddleware\PageCache->handle() /vagrant/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php:47 0.0555 1783144 7. Drupal\page_cache\StackMiddleware\PageCache->pass() /vagrant/web/core/modules/page_cache/src/StackMiddleware/PageCache.php:78 0.0555 1783224 8. Drupal\Core\StackMiddleware\KernelPreHandle->handle() /vagrant/web/core/modules/page_cache/src/StackMiddleware/PageCache.php:99 0.1101 2086200 9. Drupal\Core\StackMiddleware\Session->handle() /vagrant/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php:47 0.2719 2243832 10. Symfony\Component\HttpKernel\HttpKernel->handle() /vagrant/web/core/lib/Drupal/Core/StackMiddleware/Session.php:57 0.2719 2244688 11. Symfony\Component\HttpKernel\HttpKernel->handleRaw() /vagrant/web/vendor/symfony/http-kernel/HttpKernel.php:62 0.4059 4307336 12. call_user_func_array:{/vagrant/web/vendor/symfony/http-kernel/HttpKernel.php:139}() /vagrant/web/vendor/symfony/http-kernel/HttpKernel.php:139 0.4059 4307536 13. Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() /vagrant/web/vendor/symfony/http-kernel/HttpKernel.php:139 0.4059 4307784 14. Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext() /vagrant/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php:97 0.4063 4319720 15. Drupal\Core\Render\Renderer->executeInRenderContext() /vagrant/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php:124 0.4063 4320168 16. Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() /vagrant/web/core/lib/Drupal/Core/Render/Renderer.php:574 0.4063 4320216 17. call_user_func_array:{/vagrant/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php:123}() /vagrant/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php:123 0.4063 4320912 18. Drupal\Core\Controller\FormController->getContentResult() /vagrant/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php:123 0.5736 5691264 19. Drupal\Core\Form\FormBuilder->buildForm() /vagrant/web/core/lib/Drupal/Core/Controller/FormController.php:74 0.5969 7997376 20. Drupal\Core\Form\FormBuilder->processForm() /vagrant/web/core/lib/Drupal/Core/Form/FormBuilder.php:314 0.6039 8157464 21. Drupal\Core\Form\FormSubmitter->doSubmitForm() /vagrant/web/core/lib/Drupal/Core/Form/FormBuilder.php:583 0.6039 8157576 22. Drupal\Core\Form\FormSubmitter->executeSubmitHandlers() /vagrant/web/core/lib/Drupal/Core/Form/FormSubmitter.php:51 0.6039 8158944 23. call_user_func_array:{/vagrant/web/core/lib/Drupal/Core/Form/FormSubmitter.php:111}() /vagrant/web/core/lib/Drupal/Core/Form/FormSubmitter.php:111 0.6039 8159088 24. Drupal\system\Form\CronForm->submitForm() /vagrant/web/core/lib/Drupal[...]






Multiple taxonomy terms not importing to field from CSV

Thu, 12 Apr 2018 01:52:59 +0000

I have a typical situation where I have several taxonomy terms, two taxonomies. I would like to make multiple entries from a CSV file to associate with that field (example: node row 1 has field_term_a with term ID 1, 2, 3 in it and field_term_b with term ID 5, 6, 7). field_term_a and field_term_b are both set as permitting multiple references.

I can consistently get a *single* term on the field_term_a (eg the first term ID attaches correctly).

I tried formatting the cells by both the text of the term and the ID of the term. Also made all terms inside Drupal before importing.
1, 2, 4, -or more precisely they had quotation marks on Excel export like: "1, 7, 21, ","24, 13, 15, 16, 25, 29, 28, "
1 2 4
termname_a, termname_b, termname_c

With the first two the mapping is set reference by term id, and the latter with the reference by name.

Using the ID codes without commas gives me only the first term in the field. I couldn't find a spot in the code that was exploding them on the comma-space character pair so I'm not sure that is the right format.

I looked around at EntityProcessorBase.php, TermProcessor.php and EntityReference.php and didn't catch any good clues about this.

Workaround

Probably going to use views_bulk_operations and views_bulk_edit to insert terms manually, it should let me add individual terms to multiple nodes 'additively' so to speak.




Target fields in content type unavailable

Thu, 12 Apr 2018 00:58:56 +0000

I've noticed that some fields in a chosen content type are not available as Target Fields in the mapper. In one case the Video Embed field is not listed as a Target. I'd read some time ago that only "core" fields were supported as Target fields. Is this still this still the case? Thanks in advance The Alpha version of Feeds 8 is working nicely and your work on this project is very much appreciated.




Feeds doesn't import remote images which have no proper extension (such as ASP)

Sun, 08 Nov 2015 00:55:39 +0000

I've the following URL of image file which I want to import it:

http://media.resales-online.com/live/ShowImageXML.asp?SecId=ckzbxwfvludjkrl&Id=P1&ImgId=X1014122&z=1446224711&.jpg

However the problem is that this file doesn't have content-disposition (so I can't use the patch from #1104378-8: Allow passing custom filename to FeedsParser.inc::getFile()), so my filename is get saved as ShowImageXML.asp which fails with:

notice feeds Invalid enclosure http://media.example.com/live/ShowImageXML.asp?SecId=x&Id=P2&ImgId=y&z=z...
error Feeds exception 'PDOException' with message 'SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://images/ShowImageXML.asp' for key 'uri''

I think this is related to #1482530: Support remote files in file field., but I'm not sure.

So I guess the solution would be ASP extension to allowedExtensions, but it didn't work. It seems it works for other feeds which have proper JPG extension (like this).

My FeedsEnclosure looks like:

FeedsEnclosure::__set_state(array(
   'mime_type' => 'image/jpeg',
   'allowedExtensions' => 'png gif jpg jpeg asp',
   'safeFilename' => 'ShowImageXML.asp',
   'value' => 'http://media.resales-online.com/live/ShowImageXML.asp?SecId=ckzbxwfvludjkrl&Id=P1&ImgId=X1014122&z=1446224711&.jpg',
))

I've tried to use tamper plugin, but upon changes, the URL gets broken and I've got 404.

So I think the solution would be to save the file based on its mime type?




An error occored while fetching Test Feed.

Fri, 23 Sep 2016 16:15:00 +0000

FYI, there is a typo on line 30 of FeedClearHandler.php, occored->occured.

I think I have everything set up, but not sure since there is little info on doing this, and when I try to import I get:

An error occored while fetching Test Feed.
Please continue to the error page

An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: /batch?id=23&op=do_nojs&op=do
StatusText: Internal Server Error
ResponseText: {}

In dblog messages:

InvalidArgumentException: sink must not be a boolean in GuzzleHttp\Client->applyOptions() (line 372 of .../vendor/guzzlehttp/guzzle/src/Client.php).

Thank you.




How to import nodes with term hierarchy?

Thu, 29 Mar 2018 08:24:24 +0000

Hello,

Is there a way to import nodes through Feeds module and save "term hierarchy" - entire term lineage?

The whole import works just fine, but the term hierarchy does not!

Thank you!




\Drupal\feeds\Plugin\Type\Target\TargetInterface docs

Thu, 12 Apr 2018 09:03:59 +0000

Here's a patch that adds missing documentation to \Drupal\feeds\Plugin\Type\Target\TargetInterface

Do you think setTarget could also be updated to take FieldTargetDefinition instead of EntityInterface or does it do unnecessary code breaks?




PHP 7.2: Deprecated function: assert(): Calling assert() with a string argument is deprecated in Drupal\feeds\Plugin\Type\PluginBase->__construct()

Thu, 29 Mar 2018 08:06:40 +0000

Deprecated function: assert(): Calling assert() with a string argument is deprecated in Drupal\feeds\Plugin\Type\PluginBase->__construct() (line 50 of /data/www/default/openbooking/web/modules/contrib/feeds/src/Plugin/Type/PluginBase.php)

source code:
assert('isset($configuration["feed_type"]) && $configuration["feed_type"] instanceof Drupal\feeds\FeedTypeInterface');




Deleted Target Field Throws Fatal Error

Wed, 22 Feb 2017 14:53:09 +0000

If a field which was set as a target in the mapping for the feeds type is then deleted then the mapping page throws a fatal error so you can't fix it in UI.

Steps to reproduce:
Enable Feeds
Add a field to a content type
Add a feed with the Node Processor and Content Type with the new field from above
Add a mapping to the new field and save
Delete the field
Reload the mapping page and despair!




Expire feature deletes all imported items

Tue, 20 Sep 2016 13:52:19 +0000

Hi,
The expire checkox does not work. Below is the piece of code which delivers the id's which need to be expired.

   return $this->queryFactory->get($this->entityType())
      ->condition('feeds_item.target_id', $feed->id())
      // ->condition('feeds_item.imported', REQUEST_TIME -1, '<')
      ->execute();
  }

So actually there is no condition hence all node get deleted for the selected feed.
Also there should be a created field in schema and nodes should be expired based on created date of node not imported or we could add an option for both.

Thanks




Is there a way to Skip row processing which matches some condition

Tue, 26 Jul 2016 06:58:01 +0000

Let's say I have a row in CSV, which has a value in the column that will indicate to me that this row shouldn't be imported.

Is there a way to set some kind of indicator that will prevent the row from being saved?




Add help page

Fri, 13 Apr 2018 20:38:00 +0000

The help page says "No help is available for module Feeds.". (/admin/help/feeds)

We need to update the Page with content of following headings:

  • About
  • Terminology
  • Uses

(image)




[Policy, No patch] Consider adopting parts of migrate system in the underlying parts of feeds

Fri, 18 Aug 2017 15:43:24 +0000

I'm looking into the migrate module and entity imports, and the current state of contrib on that matter.

I've looked into the dev version and it looks like it's an independent solution. Are there any views to base Feeds on Migrate, or is it going to continue on an independent path? Any decisions made on features etc. based on your surveys? We want to push development forward and happy to collaborate on development.




Allow NULL values in mapping

Tue, 10 Apr 2018 05:30:30 +0000

Problem/Motivation

Currently FeedsParser::getSourceElement() relies on isset() to determine if a mapping / value is present. However, if the mapping value is set to NULL this fails and re-sets the value from NULL to ''.
This can lead to trouble in case the set property is validated afterwards and an empty string doesn't pass - while NULL would.

Proposed resolution

Use array_key_exists() instead isset() - the former doesn't rely on the the value of the array item at all.

Remaining tasks

Reviews needed.

User interface changes

None.

API changes

Could lead to change in behaviour where NULL values now return NULL instead an empty string.

Data model changes

None.




[meta] Feeds 8.x roadmap

Thu, 04 Apr 2013 02:18:50 +0000

Plan overview The next step for working towards a stable D8 release is fixing the UI for the CSV and XML parsers and port Feeds Tamper to D8. This will result into the first alpha release of Feeds. See #2917928: Plan for 8.x-3.0-alpha1 release. After that, the plan is to investigate if some Feeds components could be replaced with Migrate components. The benefit of using Migrate components is that it will decrease the amount of code to be maintained in Feeds and at the same time leverage from improvements in Migrate. If an integration with Migrate seems doable, this will result into a 7.x-4.x branch. The 7.x-3.x branch will likely get abandoned after some time in this case. After all this has happened, remaining features from the 7.x-2.x version need to be ported. Among more, these are: Unpublish/delete items that have been removed from the sourceD7 issue: #1470530: Unpublish/Delete nodes not included in feed. Multilingual supportD7 issue: #1183440: Multilingual Feeds - Make field import language-aware. Allow any target to be uniqueD7 issue: #661606: Support unique targets in mappers. Feature to only update items (not inserting new)D7 issue: #1286298: Don't create new items, only update existing. Based on the survey results we will also aim for the following: Support for huge data files Chained imports Rollback imports Keep same content ids as in the source Analytic report of import process And my personal favorite: Feeds Preview. Feeds 8.x M1 - An usable Feeds version for at least basic usage Goal: have an usable CSV parser, an usable XML parser and a working Feeds Tamper module with at least one Tamper plugin. See #2917928: Plan for 8.x-3.0-alpha1 release. Preparations #2847682: Drupal 8 survey Feeds/Migrate usage and functionality #2917405: UI designs Remaining tasks #2669974: Implement the CSV parser UI #2917924: Implement the XML Xpath parser UI #2917926: Port Feeds Tamper to D8 #2917925: Add a base class for Feeds functional tests Feeds 8.x M2 - Migrate integration Goal: investigate if Migrate components can be used in Feeds. If Migrate integration seems doable, a 8.x-4.x branch will be opened. See #2903175: [Policy, No patch] Consider adopting parts of migrate system in the underlying parts of feeds. Feeds 8.x M3 and beyond Goal: stable release #2383305: [meta] Add UI tests. Other automated tests. Porting remaining features from the D7 version. Implement features that have been marked important on the survey results. #2141945: Simulating a feed import Testing and bugfixing. Stable D8 release. [...]



Munging filename during import generates an unintuitive file not found exception.

Wed, 26 Aug 2015 09:46:59 +0000

Problem/Motivation

I have been tasked with organizing a migration of the data from an external party to our internal platform for our customer. This includes user profiles, which contains the resume of the user. The principle of migrating files is easy and works (map the source column to the file:uri subfield, ensure the URI is correct before it's being processed and the file already exists), despite this we encountered a score of exceptions about files not being found. Upon comparing the mentioned filename with the actual file, there always was an extra underscore in the exception (for example, filename testfile.with.extension.doc showed up as testfile.with_.extension.doc in the exception). Debugging led me to the function file_munge_filename, which indeed adds an underscore suffix to portions of the filename between dots that are shorter than six characters. This is called in FeedParser.inc function getSafeFilename (last statement before the return), as part of FeedsEnclosure's getFile. However, it is not checked whether the safe file name actually exists or whether it's still the same.

In short:
- Feeds importer with file field
- Import a file with name like 'testfile.with.extension.doc'
- Exception being shown with contents similar to 'testfile.with_.extension.doc not found'

Proposed resolution

At the very least, this specific case warrants a specific exception to be thrown, in order to clearly inform the user that the file name got munged and to rename the file in order to successfully import this file.




Plan for Feeds 7.x-2.0-beta5 release

Sun, 24 Sep 2017 12:53:21 +0000

Follow-up to #2826403: Plan for Feeds 7.x-2.0-beta4 release Issues of interest: Cron issues There are a lot of reports about imports that do not work during cron. To add more insight to the problem, a feature to see the next import time was added to 7.x-2.0-beta4. Perhaps this already fixes a part of the cron issues. We hope to fix a few other cron issues by running the imports during cron as either user 1 or the user who created the feed node.#2541944: Switch to feed author or user 1 during imports (taxonomy mapping does not work with cron)#1995728: [META] Cron import not working on 7.x-2.0-alpha8 and later Validation Feeds 7.x-2.0-beta4 has become stricter on validating imports to minimize SQL errors on imports. It also validates importers themselves. We want to extend validation by no longer allow attaching two importers to the same content type and by locking some settings when there is already imported data.#2474589: Changing processor wipes mappings with no warning#2640108: Display an error message when attaching an importer to a content type that is already in use by an other importer Aborted CSV import There is one big issue left related to imports that need multiple cron runs to complete: a source to import from is stored as a temporary file, which is removed after 6 hours. If the import hasn't been completed by then, the import will be aborted and the importer will be stuck in a locked state.#1029102: Importing Large CSV document (downloaded and processed in the background) Other issues#2922268: Standalone import works, cron import has field validation errors#2918840: Field validation errors in item#2911711: FeedsHTTPCache class could not be found: 7.x-2.0-beta4 database update can only run if Feeds is enabled [...]



Validation failure for body text filter format due to lack of permission

Mon, 18 Apr 2016 19:21:43 +0000

First, set up a feed import. Make sure to map a field to the body of the node type you're importing into. Set the filter format to a filter format that's has restricted access and is not available to anonymous users.
Now, import you data using the UI. Everything works fine.
If however you try the same import using drush, you get the following error:

The content [node title] failed to validate with the error: The value you selected is not a valid choice. Please check your mappings.




Create custom ECK entities via Feeds

Tue, 03 Apr 2018 14:04:38 +0000

At now Feeds module provide only User, Term and Node entity types in import targets (Processors).

Will be good to import all other custom entity types, defined in Drupal, such as ECK Entities, Commerce Products, etc.

Is this possible without hard rewrite of module?




"Auto detect feeds" feature causes the validation error "Invalid feed URL" for certain urls

Sun, 18 Mar 2018 12:44:03 +0000

Problem/Motivation

For downloading from certain urls, the "Auto detect feeds" feature from the Download fetcher causes issues. The error message "Invalid feed URL." is displayed and importing from that url becomes impossible when that option turned on.

This is the case for:

  • CSV files
  • "tradedoubler" url's

The bug is either in \Drupal\feeds\Feeds\Fetcher\Form\HttpFetcherFeedForm or in \Drupal\feeds\Utility\Feed. The feed class looks for common syndication urls and returns false when it finds nothing. I suspect that HttpFetcherFeedForm then wrongly assumes the url is wrong.

Proposed resolution

To be determined.

Original report by Dennis Cohn

I'm testing out the 8.x-3.x-dev version and I can't use any url's that affiliate.tradetracker.com is generating.
Anyone else who's bumped into this error? I want to use the feeds module, to import the data every night.

The .csv is generating when you're following the link http://pf.tradetracker.net/?aid=******&encoding=utf-8&type=csv&query=mal...

Am I missing some configuration in the feeds module?




[meta] Add UI tests.

Thu, 27 Nov 2014 14:01:36 +0000

Problem/Motivation

Only a small part of the UI is covered with tests.

Remaining tasks

Tests needs to be written for the following cases:

Feed type

  • The user must be able to view a list of all feed types.
  • The user must be able to add a new feed type.
  • The user must be able to edit an existing feed type.
  • The user must be able to delete an existing feed type.
  • The user must be able to configure each fetcher, parser and processor (needs tests per Feeds plugin).

Mapping form

  • The user must be able to specify custom sources. (Covered by \Drupal\Tests\feeds\FunctionalJavascript\Feeds\Parser\CsvParserTest)
  • The user must be able to choose from predefined sources.
  • The user must be able to remove mappings.

Feed

  • The user must be able to view a list of all feeds.
  • The must be able to add a new feed. (Covered by \Drupal\Tests\feeds\FunctionalJavascript\Feeds\Parser\CsvParserTest)
  • The user must be able to edit an existing feed.
  • The user must be able to delete an existing feed.
  • The user must be able to perform an import with each fetcher, parser and processor (needs tests per Feeds plugin).
  • The user must be able to delete all imported items from a feed.
  • The user must be able to unlock a feed.
  • The user must be able to add a subscription to a feed (PubSubHubbub).

There could be more cases for which an UI test would be useful.

Note: tests involving batches must be implemented in a functional test. At the time of writing batches implemented in a functional javascript test can cause random test failures (see #2938500-18: Add UI tests for the CSV parser).

Original report by h3rj4n

I'm not able to create a feeds_importer entity. I create a test that failes because of this. The entity requires a bundle to be saved.




Import problem for some images

Fri, 30 Mar 2018 20:15:28 +0000

Hi. I encountered the following problem of importing csv files with the help of feeds: some images are not imported, but if I change at least one character in the image name, then it appears when you re-import it. If I return the previous name and import again, also successfully. I attach a tamper plugin on the image field, which replaces the underscore character with the underscore character. After that, most of the problem pictures were imported, but still not all (although there are also underscores in their names). Can anyone help me? Thank you very much.
Version feeds - 7.x-2.0-beta3.




[META] Cron import not working on 7.x-2.0-alpha8 and later

Wed, 15 May 2013 18:34:05 +0000

Problem/Motivation Under certain circumstances content gets imported via a manual import, but not when running cron. Possible causes: Insufficient permissions, as cron is ran as anonymous user.#2541944: Switch to feed author or user 1 during imports (taxonomy mapping does not work with cron) An incompatible combination of settings: "Periodic import" and "Import on submission" both turned off and using the standalone form.#2445477: Process in background not working with certain combination of settings Proposed resolution Some causes are known (see above), but likely not all. The problem has been reported in multiple issues, so these issues should be sorted out to gather clues. When the other causes are known, it would be best if the bug(s) could be replicated with an automated test. This makes it easier to find out if a proposed fix really fixes the issue or just partly. Remaining tasks Sort out the issues that reported this problem (and close duplicates). Find the cause of the issue. Replicate the issue with an automated test, if possible. Propose a fix for the issue. Temporary Solution The temporary solution is to manually kick off the feed import with a custom hook_cron(). For feeds not attached to nodes, the code will look like this: function MODULE_NAME_cron() { $name = 'FEED_NAME'; $source = feeds_source($name); $source->import(); } For feeds that are attached to nodes, you'll need to load a list of node ids then run the import on each one: function MODULE_NAME_cron() { $node_type = 'CONTENT_TYPE'; $feed_type = 'FEED_NAME'; $query = new EntityFieldQuery(); $query->entityCondition('entity_type', 'node'); $query->entityCondition('bundle', $node_type); $result = $query->execute(); $feed_nids = array_keys($result['node']); foreach ($feed_nids as $feed_nid) { $source = feeds_source($feed_type, $feed_nid); $source->import(); } } Related issues These issues may be reports of the exact same problem: #1553190: data not import during cron run #1716084: Cron fails on import of csv from web published csv from a google spreadsheet #2305319: Feeds only importing occasionally #2450365: Importer is not rescheduled for new sources when using attach to node #2462475: News Feeds does not import all feeds? #2174841: Call FeedsSource::schedule() after manual import. #2059677: Feed using wrong importer when import attempted at Cron D6 #1873812: Feeds not importing on cron #1510432: Cron job doesn't import all feed items. These issues look like a different problem, but are related to cron too: D6 #927782: No new items on cron (issues with job scheduler configuration) #1029102: Importing Large CSV document (downloaded and processed in the background) #2049561: Unable to import large document during cron #2445477: Process in background not working with certain combination of settings #2385521: Broken image if run by cron #2275893: Process for non-periodic imports will get stuck #1558118: Process in background does not queue cron task for import #1947288: job_scheduler always set for import in the past #1994876: Fetcher with batch process does not fully-import during cron #2513802: "Import on submission" stucks at 56 % Closed cron issues: #2313267: Feeds cron dependa[...]



Feeds importing inconsistent numbers.

Wed, 21 Mar 2018 14:19:24 +0000

I have a feed on another site of 100 users info that simply imports to another site and shows them as staff members with a custom content type. The feed import works great and is a stand alone form with Periodic import turned on. Cron gets run with a crontab on the web server every hour.

The issue is when I look at the view total, sometimes it's 100 and a fews hours later it's 50 or 0 or 1 and then back to 100 sometime. NO clue why this is happening.




Views relation with feed entity and content feed

Fri, 16 Mar 2018 17:07:51 +0000

Hi. First of all thanks for the work done so far with the module feeds.

I have tried to created a view of the content feed with feed entity, but I was unable to find the relation.

Is it possible that feeds are not yet ready to work with views?

Thanks




Add a target for path alias

Tue, 27 Feb 2018 19:58:38 +0000

Two interconnected issues here:

First, if you import content to a content type with an autopath pattern set, the created nodes do not get the auto generated URL. E.g., they have node/###.

This seems to be because there is no 'path' entry in the values used to create the new entity. This means a test is the path field exists done by the pathauto entity hooks fails. A value is needed here to trigger the pathauto autogenerate process.

Second, item is that the path target (URL Alias) is not available when defining a map for a content type with a pathauto pattern set.

Traced this down to the path field being marked as Read Only by pathauth. It seems like the path target code should override the FieldTargetBase::targets method to allow this to be available. This would allow non-autogenerated paths to be imported.

I'm not sure the best way to approach this problem, so I don't have a patch for this.




Assign author of imported nodes

Sat, 05 Dec 2009 19:09:13 +0000

I am looking at using Feeds to import CSV data as nodes and wanted to assign all the nodes coming from a particular configuration to a specific user.

Seems this can be achieved relatively simply by adding a variable through FeedNodeProcessor.inc. A variable in the configDefaults() function, a field (with autocomplete) to the configForm(), and pulling that value out and assigning to node at the process() function.

I've attached a file that shows the final UI.

If this makes sense as functionality to have within Feeds and the method is ok (or could be improved) happy to roll a patch.

One complication might be that if one wants to assign different nodes to different users at the node import stage, based on a value in the feed through a mapper we would have to override the user assigned through the method described here (or resort to this user as the default one).




Timezone handling broken

Fri, 20 Oct 2017 16:07:59 +0000

I just updated from beta 3 to beta 4. No other changes were main to the website.

I have a feeds importer using JSONpath to convert JSON data to nodes. One of the mappings is from a UTC date to a date field. With beta3, Feeds correctly imported a date with the format YYYYMMDDTHHMMSSZ (example: 20161001T040000Z) to a date field. Now with beta4 -- just with this one change to the site -- Feeds is returning the following error and failing to import:

DateTimeZone::__construct() [datetimezone.--construct]: Unknown or bad timezone (Z)

Z is not a bad time zone indicator. Time zone can be Z or some +/- a certain number of hours. Since nothing else about the site changed, I have to assume Feeds is the problem. I've also seen this issue on several other website I run with date imports.




Add a base class for Feeds functional tests

Sun, 22 Oct 2017 14:53:28 +0000

There should be a base class for functional tests in Feeds, as tests to be written for #2669974: Implement the CSV parser UI and [#] will likely have some overlap.
The base class should be called FeedsBrowserTestBase and be added to feeds/tests/src/Functional.

The class FeedsWebTestCase from the D7 version of Feeds can be used as reference.

Requirements

  • Feeds module should be enabled.
  • The setUp() method should add a content type called 'article'.
  • The setUp() method create an user with Feeds admin privileges.
  • There should be a method for adding a feed type.
  • There should be a method for adding mapping.
  • There should be a method for adding a feed.
  • There should be a method performing an import.

Optional

  • Port other methods from the FeedsWebTestCase class from the D7 version of Feeds.



Importing users with password

Wed, 07 Mar 2018 14:49:10 +0000

I have a case which i need to import users with password, feeds currently does not support importing passwords.
Can you please help me or guide me how to make this done.




Standalone import works, cron import has field validation errors

Thu, 09 Nov 2017 17:09:38 +0000

Since installing beta4 imports on cron runs fail with field validation errors. But if i run the importer from a standalone instance nodes are created. Log entry for failure on cron run (I have removed the sender's name):

2017-11-09 11:50:28 	2017-11-09 11:50:01 	Failed importing 3 nodes. 	error
2017-11-09 11:50:28 	2017-11-09 11:50:01 	Field validation errors in item 'Last call for directory corrections.':
Original item array( 'imap_uid' => 4685, 'body_text' => "promote: 0\r\nsticky: 0\nI am attaching the most up-to-date draft. Please check it and let me know\r...", 'body_html' => '
I am attaching the most up-to-date draft. Please check it and let me know if you ...', 'authenticated_uid' => '222', 'date' => 'Thu, 9 Nov 2017 11:17:27 -0500', 'Date' => 'Thu, 9 Nov 2017 11:17:27 -0500', 'subject' => 'Last call for directory corrections.', 'Subject' => 'Last call for directory corrections.', 'message_id' => '', 'toaddress' => 'cybershtetl ', 'to-mailbox' => 'cybershtetl', 'to-host' => 'e-kehila.org', 'fromaddress' => 'XXXXXXXX ', 'from-mailbox' => 'XXXXXXX', 'from-host' => 'gmail.com', 'reply_toaddress' => 'XXXXXXXe ', 'reply_to-mailbox' => 'XXXXXXXXX', 'reply_to-host' => 'gmail.com', 'senderaddress' => 'XXXXXXX ', 'sender-mailbox' => 'XXXXXXXXX', 'sender-host' => 'gmail.com', 'Recent' => '', 'Unseen' => 'U', 'Flagged' => '', 'Answered' => '', 'Deleted' => '', 'Draft' => '', 'Msgno' => ' 458', 'MailDate' => ' 9-Nov-2017 11:18:02 -0500', 'Size' => '274267', 'udate' => '1510244282', 'status' => '0', 'comment' => '0', 'moderate' => '0', 'og_public' => '1', 'body_original' => '
I am attaching the most up-to-date draft. Please check it and let me know if you ...', ) Entity (object) array( 'language' => 'und', 'type' => 'kc_announcement', 'changed' => 1510246201, 'created' => 1510246201, 'is_new' => TRUE, 'status' => '0', 'promote' => 0, 'sticky' => 0, 'uid' => '222', 'revision' => FALSE, 'comment' => '1', 'expire' => '2017-12-21', 'log' => 'Created by FeedsNodeProcessor', 'title' => 'Last call for directory corrections.',



Access denied on /admin/structure/feeds/import (trying to import an importer)

Wed, 21 Feb 2018 11:56:09 +0000

From what I understand of various issues there is an UI for importing feeds importers. After some digging I found that the path for this is /admin/structure/feeds/import
However when I visit that URL I am getting a access denied error. I am not user 1, but I do have ALL permissions checked on my development account/server. What's going on?

Marking as minor as it does not impact core feeds functionality.




Plugin path seems off-default

Sun, 25 Feb 2018 12:25:55 +0000

All 8.x modules I worked with put their Plugins under "src/Plugin/[module_name]/[plugin_name]". Feeds instead does /src/Feeds...

It would be nice if this could be changed to fit expectations, wouldn't it?




Provide feeds type fields as mapping sources

Mon, 25 Sep 2017 14:00:54 +0000

Hi,

I am working on this feature, very soon i will update the patch...
Needs suggestions

The scenario is - I needs to import content under content type Faq so for that I have created one csv with question and answer only..
But I have some other entity reference fields under faq for this, I created same entity reference fields under the feeds type (Manage Fields) and i am importing faq csv, that should be import along with reference field value.

For This -

  • Created Faq Content type with some entity reference fields, question and anwser field
  • Created Feeds Type and same entity reference field as having in faq
  • Mapping fields created under the feeds type with faq fields
  • Created csv with question and answer value
  • On feed import page filling the import form and uploading csv file
  • After import, faq getting created with all the data provided in csv as well as terms

Thanks




Feeds Processor, give warning if the parameter of hash function is not an array

Mon, 22 Aug 2016 10:48:11 +0000

I am using feeds module with cron for periodic nodes update and i found this warning in the CRON report

Warning: array_intersect_key(): Argument #1 is not an array in FeedsProcessor->hash()

This warning is coming when the Arugemnt#1 of the hash() function is not an Array.
there is not check for the parameter type and this function is trying to run regardless of the type of the parameter.




Attach multiple importers to one content type (in D7)

Thu, 14 Apr 2011 21:03:18 +0000

Problem/Motivation

http://drupal.org/node/634462

Exact same as the above feature request which looks like its solved for d6.

For convenience here is copy pasted issue

I can assign more than one feed to a content type by setting the 'Attach to content type:' to the same content type for two different feeds. However, when I go to create my node, I only see one textfield to enter in the feed URL. I should see two.

My use case: I'm using the project.module to detail projects that I maintain on Drupal.org and I want to attach a few feeds to the node. The CVS commit history, release history, issue queue etc. So, when I go to create a new project node I should see textfields for each associated feed, allowing me to associate all the feeds with the project (I'm using the data storage backend).

Proposed resolution

Node form alters are modified so that config forms of multiple importers are displayed right there on node form if multiple importers have been added to same node type.

Remaining tasks

  1. Write more tests
  2. Review the code
  3. Manual testing

User interface changes

Multiple importer config forms are now shown on node edit forms.

API changes

-




Drupal 8 survey Feeds/Migrate usage and functionality

Sat, 28 Jan 2017 16:17:07 +0000

We are working on porting Feeds to Drupal 8. A lot of functionality is duplicated between the Migrate module and the Feeds module. In this survey we would like to know how you use these and similar import/export modules and what functionality you like but still miss in Drupal 8.

We will use the results of this survey to prototype and plan import/export functionality for Drupal 8.
We did not decide yet if this will be done in the Feeds module or if we take a different approach.

Please take the survey.
https://docs.google.com/forms/d/e/1FAIpQLSe9zjZGM7chX3lSZAxnySygClCIlWrj...




Add support for unique fields to be unique site wide

Thu, 19 Apr 2012 09:35:28 +0000

I have several feeds from the same publication database with different authors and on some publications they are coauthors.

However the GUID is unique, and should according to my understanding prevent Feeds from saving the publication twice.

Is this a bug in Feeds, or have I misunderstood the use of unique GUIDs.

Svend




Core Telephone Field

Fri, 16 Feb 2018 19:56:17 +0000

Noticed that there wasn't an option to map the core telephone fields to a feed.

Going off of what was already there for Core fields something this enabled it for me.

Not sure on what type of validation would need to take place there or if that's all on the content source.




Filefield Paths incompatibility with "Skip if file exists" option

Sat, 10 Feb 2018 23:24:57 +0000

Currently Filefield Paths is incompatible with "Skip if file exists" option when importing files via feeds. Having both available would be definitely useful.

Thanks in advance for the feature request consideration.




Enclosure support in Feeds

Tue, 26 Oct 2010 06:56:19 +0000

I'm currently setting up a Feeds importer using the HTTP fetcher, Common syndication parser, and Node processor, but it doesn't look like RSS enclosures (or MediaRSS) are supported at all in FeedsParser, its subclasses or _parser_common_syndication_RSS20_parse().

Is enclosure/MediaRSS support in the pipeline, or is there another project that adds enclosure/MediaRSS support? if not, I'll see what I can do to provide a patch for this.




Fix coding standards

Sun, 21 Sep 2014 06:55:40 +0000

Hi I have reviewed the Feeds 7.x-2.0-alpha8 and found its not following the drupal coding standard. So I have planed to create a patch on that and will fix all the possible issues. Best Practices (Coding Standards) Coding standard issues on feeds 7.x-2.0-alpha8 feeds.rules.inc Line 40: String concatenation should be formatted with a space separating the operators (dot .) and the surrounding terms [style_string_spacing] $info['feeds_import_'. $importer->id] = array( Line 55: String concatenation should be formatted with a space separating the operators (dot .) and the surrounding terms [style_string_spacing] $info['feeds_import_'. $importer->id]['variables'][$entity_type]['bundle'] = $processor->bundle(); feeds_ui.admin.inc Line 1135: Using eval() or drupal_eval() in your module's code could have a security risk if the PHP input provided to the function contains malicious code. (Drupal Docs) [security_19] eval($form_state['values']['importer']); common_syndication_parser.inc Line 88: do not use mixed case (camelCase), use lower case and _ [style_camel_case] function _parser_common_syndication_atom10_parse($feed_XML) { Line 95: do not use mixed case (camelCase), use lower case and _ [style_camel_case] $base = $feed_XML->xpath("@base"); Line 102: do not use mixed case (camelCase), use lower case and _ [style_camel_case] $parsed_source['title'] = isset($feed_XML->title) ? _parser_common_syndication_title("{$feed_XML->title}") : ""; Line 104: do not use mixed case (camelCase), use lower case and _ [style_camel_case] $parsed_source['description'] = isset($feed_XML->subtitle) ? "{$feed_XML->subtitle}" : ""; Line 106: do not use mixed case (camelCase), use lower case and _ [style_camel_case] $parsed_source['link'] = _parser_common_syndication_link($feed_XML->link); Line 113: do not use mixed case (camelCase), use lower case and _ [style_camel_case] foreach ($feed_XML->entry as $news) { Line 189: do not use mixed case (camelCase), use lower case and _ [style_camel_case] if (!empty($feed_XML->author->name) && !$author_found) { Line 245: do not use mixed case (camelCase), use lower case and _ [style_camel_case] function _parser_common_syndication_RDF10_parse($feed_XML) { Line 261: do not use mixed case (camelCase), use lower case and _ [style_camel_case] $namespaces = $feed_XML->getNamespaces(TRUE); Line 264: do not use mixed case (camelCase), use lower case and _ [style_camel_case] foreach ($feed_XML->children($canonical_namespaces['rss'])->channel as $rss_channel) { Line 275: do not use mixed case (camelCase), use lower case and _ [style_camel_case] foreach ($feed_XML->children($canonical_namespaces['rss'])->item as $rss_item) { Line 35[...]