Magento Setup and Amazon Associates Web Service

Install and configure a Magento test server using sample data from the Amazon Associates Web Service quickly and easily. There are a number of common pitfalls that can be easily avoided and some that cannot. The process of converting information from Amazon to Magento is described briefly.

Magento made an ideal candidate for load testing due to its no cost availability and instant setup. It will be an example target of a product shopping website. There is little documentation for this software. I found the bulk of support information on the Magento forums. Once Magento is configured with the LoadStorm user IDs, virtual users will be able to browse the website both anonymously and signed-in. They will collect items in their carts randomly and some proportion of them will proceed to checkout.

The Magento setup is quite short. It requires a standard LAMP installation. After the configuration of the SQL database, there is a short wizard in order to complete the installation.

I suggest that first time users download and install the sample data. This additional information assisted me greatly in understanding the product import process. It is suggested on several sites that you export before import. In other words, install the sample data, export the sample data through Magento's export mechanism, and configure the exported data sheet with your own information. This will avoid issues with unspecified data fields as well as provide the user with information as to how Magento is storing particular fields.

It is important to plan the layout of your site before importing a single product. Based on your current product offering, build a category structure that provides the most logical browsing for the user. Unfortunately, you must identify the category by its ID number when importing products.

Once a user has reached a subcategory, you can allow specific fields to be filterable. Filterable fields will further reduce the results set available to the user. Filterable fields must be catalog input type Dropdown, Multiple Select or Price. In other words, a filterable field must be configured with a somewhat generic or consistently repeated value.

The data imported into Magento was obtained from Amazon Associates Web Service (AAWS). AAWS provides a simple web API and existing code base for retrieving product information. I did not require a significant amount of information or variability in the types of information received.

Fields Retrieved and the Corresponding Magento Attributes

The Java code retrieves the information, writes it to file in csv format, and writes the images to file using their original names. These files can be moved to the 'magento/media/import' directory on the host server to be imported into Magento using the 'Import All Products' profile.

The products were attempted to be imported in 3 blocks. The block sizes were 3000, 1214, and 4000. Four thousand is the maximum number of records retrievable from AAWS in a single search query. This limit is caused by the fact that AAWS does not allow more than 400 pages of 10 items per page to be retrieved in any one search.

The time required to import these blocks was significant. The 4000 item block failed at importing ~1500 in about 3 hours due to an apparent timeout. There appears to be a significant problem with the speed of the import function. Read the forum posts on this topic below(Must be registered with Magento).

http://www.magentocommerce.com/bug-tracking/issue?issue=2068
http://www.magentocommerce.com/boards/viewthread/9002/P0/
http://www.magentocommerce.com/bug-tracking/issue?issue=2097

The defect was discovered in March, 2008. It is odd that the defect has no been fixed. Since my purposes required only a one time import, I simply deleted the multiple store views to improve performance. This gave a slight speedup to the import process. I made no other changes. The performance enhancements mentioned in the blog posting below may be applied in a future test case of LoadStorm.

http://www.magentocommerce.com/blog/performance-is-key-notes-on-magentos-performance

The issues listed below can be easily avoided if they are known:

• The quantity field does not allow values greater than or equal to one million. The records will import, but there will be an unspecified error indicated in the import window.
• Several different issues can cause products to fail to show after import. Ensure that the product has a quantity greater than zero and has an 'is_in_stock' value of 1. Furthermore, ensure that both the 'websites' and 'store' fields are configured correctly for your particular installation.
• Some spreadsheet programs will save the price field with a currency character. Magento will not read the currency character correctly and will throw an error. These must be formatted as text before conversion into csv in order to avoid the character being written. Overall, the import process was not friendly when all the fields were not surrounded by double quotes. OpenOffice will not automatically surround every field when saved as csv format. In order to force this behavior, select all cells and format them as the 'Text' type. This behavior was tested on OpenOffice and not Excel. Check your output file if you are unsure of the results.
• Magento's built-in mechanism for deleting products does not remove the deleted product's images. The images will remain in the Magento folder structure and are located at '/magento/media/catalog/product'. If the product previously deleted is imported back in, then its images will be imported once again as well. The older images will be renamed to '[some file]_2.[ext]'. The number will be increased as more redundant images are entered. I have had no problem deleting the old images once their corresponding product was deleted. There is a forum discussion on this here.
• I have successfully updated records by using identical 'sku' fields. In order to bulk update existing products in Magento, I used the import mechanism with the original product field values except the modified field value. The modified field value is updated correctly. I have not experimented thoroughly with this feature. I would not try this on a production site until I learned more.