Item (Magento\Catalog\Model\Product\Interceptor) with the same ID "##" already exists.

[Stylish015]

Using the in-built Product Importer we have been successfully updating our product quantities in bulk within Mangeto 2.0.5.

However since we have upgraded to Magento 2.1.0, and after a Product Import updating qty's, we get an error:
Item (Magento\Catalog\Model\Product\Interceptor) with the same ID "4" already exists.

This error shows in the Admin while trying to view products on "Products -> Catalog"

Steps to reproduce

1. Create a csv with two columns "sku" & "qty", with the existing sku's and new product qty's
2. Upload csv within "System -> Import" selecting:
3. Entity Type: Products
4. Import Behavior: Add/Update
5. The csv checks fine (no errors) and the import is successful.
6. Then in the Admin go to ""Products -> Catalog" get the error "Item (Magento\Catalog\Model\Product\Interceptor) with the same ID "4" already exists. "

Expected result

For the import to be successful.
It updates the product quantities successfully.
You can still view products in the Admin at "Products -> Catalog"

Actual result

While viewing Products in the Admin at "Products -> Catalog" get the error:
Item (Magento\Catalog\Model\Product\Interceptor) with the same ID "4" already exists.

Is this a bug?
Can it be fixed so we can view our products in the Admin?

Thanks.

[Sam-PUMP]

Exact same thing here... let me know if you find a solution

[piotrekkaminski]

One of our clients run into similar issue recently and found out there were duplicate product IDs in the database each with different Store IDs (1 and 2). They ended up deleting products with Store ID 2 as a workaround. Hope it helps.

[vishveskrish]

We are facing the same error, Is there any permanent fix ?

[GordonLesti]

We have the same problem with Item (Magento\Catalog\Model\Product\Interceptor) with the same ID "3523" already exists. if we search a product by the name Raff in the Backend of Magento 2.1.

It seems that the query is producing duplicated rows when joining cataloginventory_stock_item. This query is fetched in Magento\Eav\Model\Entity\Collection\AbstractCollection::_loadEntities. The exception will be thrown in Magento\Framework\Data\Collection::addItem cause we have two entities with the same ID.
The causing query looks like this:

SELECT `e`.*, IF(at_name.value_id > 0, at_name.value, at_name_default.value) AS `name`, `at_qty`.`qty` FROM `catalog_product_entity` AS `e`
 INNER JOIN `catalog_product_entity_varchar` AS `at_name_default` ON (`at_name_default`.`entity_id` = `e`.`entity_id`) AND (`at_name_default`.`attribute_id` = '70') AND `at_name_default`.`store_id` = 0
 LEFT JOIN `catalog_product_entity_varchar` AS `at_name` ON (`at_name`.`entity_id` = `e`.`entity_id`) AND (`at_name`.`attribute_id` = '70') AND (`at_name`.`store_id` = 1)
 LEFT JOIN `cataloginventory_stock_item` AS `at_qty` ON (at_qty.`product_id`=e.entity_id) AND (at_qty.stock_id=1) WHERE (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') AND (IF(at_name.value_id > 0, at_name.value, at_name_default.value) LIKE '%Raff%') ORDER BY `e`.`entity_id` ASC
 LIMIT 20;

The LEFT JOIN cataloginventory_stock_item ... creates the duplicate row and a GROUP BY e.entity_id`` can solve that.

We just have to find out, where we can add that in the code 😄.

[GordonLesti]

A groupByAttribute on the collection in Magento\Catalog\Ui\DataProvider\Product\ProductDataProvider::getData fixed the problem temporary.

    /**
     * Get data
     *
     * @return array
     */
    public function getData()
    {
        $this->getCollection()->groupByAttribute('entity_id');
        if (!$this->getCollection()->isLoaded()) {
            $this->getCollection()->load();
        }
        $items = $this->getCollection()->toArray();

        return [
            'totalRecords' => $this->getCollection()->getSize(),
            'items' => array_values($items),
        ];
    }

[vishveskrish]

Hi Gordon,

This is temporary fix but its not the solution as M2 adding duplicate entries on "cataloginventory_stock_item" table while importing the products.

We need to fix this issue as soon as possible.

While updating any products it is re-entering the new rows for updated products .

Thanks,
Vishves Shah

[sevos1984]

This is a known issue MAGETWO-57490, thanks for posting

[phamngsinh]

vendor/magento/framework/Data/Collection.php

line : 406

 public function addItem(\Magento\Framework\DataObject $item)
    {
        $itemId = $this->_getItemId($item);

        if ($itemId !== null) {
            if (isset($this->_items[$itemId])) {
                throw new \Exception(
                    'Item (' . get_class($item) . ') with the same ID "' . $item->getId() . '" already exists.'
               );
            }
            $this->_items[$itemId] = $item;
        } else {
            $this->_addItem($item);
        }
        return $this;
    }

become => below, wait for M2 fix
public function addItem(\Magento\Framework\DataObject $item)
{
$itemId = $this->_getItemId($item);

    if ($itemId !== null) {
        if (isset($this->_items[$itemId])) {
        return $this;
        }
        $this->_items[$itemId] = $item;
    } else {
        $this->_addItem($item);
    }
    return $this;
}

[Mahmoud-Refaat]

This problem exist duo to difference between products "website_id" value and default value in Magneto 2 Table: cataloginventory_stock_item

So do the following :
1- check the default value for your site it will be at table cataloginventory_stock
2- then compare it to the existing products at Table: cataloginventory_stock_item
in my case it was 0 while products website_id were 1 so to solve this run the following SQL
UPDATE cataloginventory_stock_item SET website_id=0

[johnverzijden]

Like above said, Magento 2 admin is not preped for diff website_id 's in the stock item table. I had to use different website_id. In case You would like to use stock per website you could preference (admin di.xml) the "Magento\CatalogInventory\Ui\DataProvider\Product\AddQuantityFieldToCollection" and add "AND {{table}}.website_id=0".

{{table}}.stock_id=1 AND {{table}}.website_id=0

[robbiefowler]

has this been fixed in post 2.1.5?

[magento-engcom-team]

@Stylish015, thank you for your report.
We were not able to reproduce this issue by following the steps you provided. If you'd like to update it, please reopen the issue.
We tested the issue on 2.3.0-dev, 2.2.0, 2.1.9

[amenk]

Got this issue after upgrading from 2.1.8 to 2.2.0 n98-magerun2 index:reindex helper for me.
I am using a custom, bit hacky, module https://github.com/Mestrona/Mestrona_BundleCustomOptions - maybe the discrepancy was caused by that - not sure. (it simply allows custom options for bundles). Just a wild guess.

[webscot]

Problem is ALIVE & WELL as of November 17, 2017 in v. 2.2.1. Used Save & Duplicate to create product #2. Product #2 never appeared even after re-indexing with cache disabled. Enabled cache, and we get this error. We also moved one category (#43) up in position. It is this #43 that is mentioned in the error report. After re-enabling cache, product #2 is now there, but still no navigation categories. Enable cache again and the error returns.

[versdivers]

I found out that my product_index_eav_decimal sometimes has multiple values for the same entity id while if i search that entity id in the attribute_decimal table or in the product table it has only 1 value. How does this table gets generated?

[versdivers]

You can see it very clear here ->

3 times 3 -> 3 store ID's , 3 times the same source_id/storeid

[itsmeit268]

@smagic39 Thankyou , it works for me

[hubertus2017]

Hello, we have magento 2.2.6 with different stores, and after hours of search only the fix in collection.php from smagic39 worked

vendor/magento/framework/Data/Collection.php

line : 406

 public function addItem(\Magento\Framework\DataObject $item)
    {
        $itemId = $this->_getItemId($item);

        if ($itemId !== null) {
            if (isset($this->_items[$itemId])) {
                throw new \Exception(
                    'Item (' . get_class($item) . ') with the same ID "' . $item->getId() . '" already exists.'
               );
            }
            $this->_items[$itemId] = $item;
        } else {
            $this->_addItem($item);
        }
        return $this;
    }

become => below, wait for M2 fix
public function addItem(\Magento\Framework\DataObject $item)
{
$itemId = $this->_getItemId($item);

    if ($itemId !== null) {
        if (isset($this->_items[$itemId])) {
        return $this;
        }
        $this->_items[$itemId] = $item;
    } else {
        $this->_addItem($item);
    }
    return $this;
}

We have the question

• what it in detail does, does it only suppress the warning?
• Is the fix safe?
• Why is it not solved for M2.2.6?

[distvan]

I can reproduce this issue in version 2.3.0 well anytime:
1.step: turn off the inventory management (Stores/configuration/catalog/inventory/product stock options/Manage stock: NO)
2. step: create a category
3. step: create a product and put into the category
4. load the category page and see the error

[oleksandrmakhno]

1. for some category the issue is reproducible on M2.2.4
2. the extreme solution with di preference for \Magento\Framework\Data\Collection is not working
3. likely it's data issue (after import for example) and can be fixed by data manipulation with some table

[distvan]

I can't reproduce with a clear M2.3.0 install but only an other existing store with some extension, maybe there are some data inconsistencies around the cataloginventory_stock_item table..... Here is a plus one record.

[amarroni]

Hi guys how are you?,

I create a very simple module with plugin inside in order to prevent this issue. https://github.com/amarroni/magento-framework-data-collection

Please is just a code to not touch the code.

Best,
Alejandro.

[pmzandbergen]

Encountered this error in Magento 2.3.1. In our case it was related to double values in the table "review_entity_summary". To check if this is the case for your database use the following SQL query:

SELECT COUNT(*) AS cnt, entity_pk_value FROM review_entity_summary GROUP BY entity_type, entity_pk_value, store_id HAVING cnt > 1;

In my opinion these tables should use a unique index to prevent this from happing (in this example a unique index over entity_type, entity_pk_value and store_id).

[pangteypiyush]

Got this in migrated Magento2.3.2 store.

[JamesAllwood]

I've also experienced this, again, down to a duplicate, but for me, it was actually an attribute that was assigned to an attribute set twice. Not sure how it happened, but wanted to record it here for reference. Table concerned was eav_entity_attribute for anyone that's having similar issues

[munkhulzii]

this issue still appears in admin area if multiple websites and different prices for each websites. There is "SELECT DISTINCT" in collection query which affects the result. In frontend it works then because I guess there is a filter with only 1 website-ID.

[jurvi]

this issue still appears in admin area if multiple websites and different prices for each websites. There is "SELECT DISTINCT" in collection query which affects the result. In frontend it works then because I guess there is a filter with only 1 website-ID.

Yea, I just bumped into this exact issue as well within pagebuilder. It's particularly nasty there because I can't modify/delete the broken content...

[jonathanribas]

We also have this issue on 2.3.7, we use Page Builder too and have several websites / customer groups.

We reproduce this issue when we use a Products widget on Page Builder. The query that is causing this issue is the following one as it returns several times the same SKU:

SELECT DISTINCT e.*, at_visibility.valueASvisibility, price_index.price, price_index.tax_class_id, price_index.final_price, IF(price_index.tier_price IS NOT NULL, LEAST(price_index.min_price, price_index.tier_price), price_index.min_price) AS minimal_price, price_index.min_price, price_index.max_price, price_index.tier_price, stock_status_index.is_salableFROMcatalog_product_entityASeINNER JOINcatalog_product_entity_intASat_visibility ON (at_visibility.row_id=e.row_id) AND (at_visibility.attribute_id = '99') AND (at_visibility.store_id= 0) INNER JOINcatalog_product_index_priceASprice_indexON price_index.entity_id = e.entity_id AND price_index.customer_group_id = 0 INNER JOINcatalog_product_entityASproductON product.entity_id = e.entity_id AND (product.created_in <= '1623718800' AND product.updated_in > '1623718800') LEFT JOINinventory_stock_5ASstock_status_index ON product.sku = stock_status_index.sku WHERE ((((at_visibility.value = '2') OR (at_visibility.value = '4'))) AND (((IFNULL(e.sku, 0) IN ('346') OR (FIND_IN_SET ('346', e.sku) > 0)) ))) AND (e.created_in <= '1623718800') AND (e.updated_in > '1623718800') ORDER BY (FIELD(e.sku, '346'));

@jurvi did you find a way to fix this issue?

[nuno-robosavvy]

We also had this issue using a module with multiple warehouses and the issue is due to a query to the database where the magento developers forgot to filter by website ID. All the others have the websiteId filtered but this specific query does not.

The file in question is /vendor/magento/module-inventory-catalog/Model/ResourceModel/AddStockDataToCollection.php

line 65 this query is built

$collection->getSelect()
	->join(
		['stock_status_index' => $resource->getTable('cataloginventory_stock_status')],
		sprintf('%s.entity_id = stock_status_index.product_id', Collection::MAIN_TABLE_ALIAS),
		[IndexStructure::IS_SALABLE => $isSalableColumnName]
	);

As you can see it does not filter the website ID, so after this I added below

$collection->getSelect()->where('stock_status_index.website_id = ? ', $this->legaycStockConfiguration->getDefaultScopeId());

Of course the $this->legaycStockConfiguration needs to be declared for that add the following:

next to all other "use" add:

use Magento\CatalogInventory\Api\StockConfigurationInterface;

define a new variable on the class

/**
 * @var StockConfigurationInterface
 */
private $legaycStockConfiguration;

Initialize this on the construct

public function __construct(
    StockIndexTableNameResolverInterface $stockIndexTableNameResolver,
    DefaultStockProviderInterface $defaultStockProvider = null,
    StockConfigurationInterface $legaycStockConfiguration
) {
    $this->stockIndexTableNameResolver = $stockIndexTableNameResolver;
    $this->defaultStockProvider = $defaultStockProvider ?: ObjectManager::getInstance()
        ->get(DefaultStockProviderInterface::class);
$this->legaycStockConfiguration = $legaycStockConfiguration;
}

btw, this code was based on code from /vendor/magento/module-inventory-catalog/Model/ResourceModel/SetDataToLegacyStockStatus.php that already takes in consideration the website id

[amitmaurya1024]

this issue also exist in magento 2.4.3-p1. Guys, any resolution for this issue ?

[kilubaka]

Hello guys, I'm also struggling with this issue. The main problem was that query:

SELECT DISTINCT `e`.*,
`at_visibility`.`value` AS `visibility`,
`price_index`.`price`,
`price_index`.`tax_class_id`,
`price_index`.`final_price`,
IF(price_index.tier_price IS NOT NULL, LEAST(price_index.min_price, price_index.tier_price),
price_index.min_price) AS `minimal_price`,
`price_index`.`min_price`,
`price_index`.`max_price`,
`price_index`.`tier_price`,
`stock_status_index`.`is_salable`,
`cat_pro`.`position` AS `cat_index_position`
FROM `catalog_product_entity` AS `e`
INNER JOIN `catalog_product_entity_int` AS `at_visibility`
ON (`at_visibility`.`row_id` = `e`.`row_id`) AND (`at_visibility`.`attribute_id` = '102') AND
(`at_visibility`.`store_id` = 0)
INNER JOIN `catalog_product_index_price` AS `price_index`
ON price_index.entity_id = e.entity_id AND price_index.customer_group_id = 0
INNER JOIN `catalog_product_entity` AS `product` ON product.entity_id = e.entity_id AND
(product.created_in <= '1638352981' AND product.updated_in > '1638352981')
LEFT JOIN `inventory_stock_2` AS `stock_status_index` ON product.sku = stock_status_index.sku
INNER JOIN `catalog_category_product` AS `cat_pro`
ON cat_pro.product_id = e.entity_id AND cat_pro.category_id IN (156)
WHERE ((((at_visibility.value = '2') OR (at_visibility.value = '4'))) AND (((IFNULL(`e`.`entity_id`, 0) IN
(SELECT `catalog_category_product`.`product_id`
FROM `catalog_category_product`
WHERE (category_id IN ('156')))))))
AND (e.created_in <= '1638352981')
AND (e.updated_in > '1638352981')
ORDER BY `cat_pro`.`position` ASC, `e`.`entity_id` DESC
LIMIT 20

was selecting products with the same entity_id but different prices, that's why it's creating duplicates.

The solution was in adding a website where query, to include products only once per load. You need to create Preference on
_beforeLoad() method in \Magento\PageBuilder\Ui\DataProvider\Product\ProductCollection, smt like this

protected function _beforeLoad()
{
$this->getSelect()->where("`price_index`.website_id = ?", $websiteId);
return parent::_beforeLoad();
}