Amazon Web Scraping: Collecting Product Data, Prices, and Reviews
Published: ≈ 14 min read
Quick Answer
An Amazon product can stay the same while its offers and customer feedback change. For a price history, repeatedly collect offers for a fixed set of ASINs, Amazon’s catalog identifiers. For review research, collect individual review records under recorded filters. Catalog details provide the product identities that connect those results.
Use one familiar product to check the output of your permitted source. Does the returned ASIN belong to the variation you selected? What is included in its price, and are individual review records present? Resolve those questions in the pilot. A mismatch at this stage is easier to resolve than an error repeated across a scheduled collection.
Key Takeaways
- Keep the chosen variation’s ASIN and marketplace together. Save size and color as a quick way to spot a mismatch.
- A discount can have conditions. Store those conditions with the amount so an analyst can tell which buyers the price applies to.
- Count the written reviews retained after deduplication. Keep that count separate from the product’s displayed rating total.
- Check the API’s output against the research task. Catalog records and review-topic summaries serve different uses.
- Use proxy geography and session settings to support a defined collection context. They do not establish permission or guarantee complete results.
Does Amazon Allow Web Scraping?
Public visibility alone does not establish permission to automate collection. Amazon.com’s Conditions of Use address this under License and Access. The license permits personal, non-commercial use. Its exclusions specifically cover collecting product listings, descriptions and prices and using automated extraction tools.
Check the marketplace and source terms for both collection and your planned storage or publication of the results. The workflow here assumes permission to collect and retain the relevant data.
Which Source Should You Use for Amazon Data?
Work back from the intended output. Catalog information for an affiliate page, a history of offer prices, and a set of review texts each call for different source capabilities.
Amazon Creators API
For an affiliate catalog integration, inspect Amazon Creators API.
- Known ASINs: GetItems retrieves their product details.
- Product discovery: SearchItems searches the catalog.
- Variation family: GetVariations supplies variation details for a parent item.
Amazon Selling Partner API, including Customer Feedback API
Seller and vendor applications have a separate route through SP-API. Customer Feedback API supplies positive and negative review topics and trends for supported products and categories. Amazon documents a weekly refresh and access through specified application roles.
Third-party scraper service or licensed dataset
Use one of your own ASINs for the supplier’s sample. A product response needs the price and shipping fields; a review response needs the comments themselves. Then establish what the bill counts and how incomplete output is charged. Storage and republication rights belong in that agreement too.
Your own authorized HTML collection
With your own collector, you decide which response to request and where to read each field. You also own the maintenance work. A moved price element can break extraction even though the listing still loads. Choose this approach where page collection is permitted, and check that the required content reaches your client.
Check the API generation when using a tutorial. Amazon’s PA-API 5 deprecation notice links to the Creators API migration guidance. Use that guidance when adapting an old request or authentication example.
Ask the supplier to distinguish an unavailable offer from retrieval and extraction failures when a JSON price is blank. The blank alone cannot establish that the item is out of stock.
What Should Each Product Record Contain?
Keep product identity separate from dated offers and individual reviews. In a hypothetical run with ten successful checks of one variation, one product row can link to ten observations of the selected offer. Reviews attach to that product without requiring copies of its full catalog details.
For a variation family, the parent reference groups related items. Match a price to the selected child’s ASIN and attributes, so that another size or color cannot silently enter the same price history.
The records have different jobs:
Product
- Identity: marketplace and observed ASIN.
- Trace: requested ASIN and source URL.
- Details: title, brand, selected size or color, and parent ASIN where supplied.
Offer snapshot
An offer row links the capture time and displayed amount to a product, seller, condition, and currency. Add shipping, availability, promotion terms, delivery destination, and the collection outcome. You can then compare two visits without losing the circumstances behind either price.
Review
Use the review identifier when available and link the row to the product and marketplace. Retain its title, text, star rating, review date, displayed variation, source URL, and capture time. Copy any purchase-verification label the source supplies. An absent label leaves that field unknown.
A destination entered in the job settings tells you what you asked for. Keep a separate field for the location the source actually showed, and leave it unknown if the response gives no location. You can retain the record, but a comparison restricted to that destination needs a confirmed value.
Save source evidence within the applicable retention rules, along with the extraction version. Use the saved response to trace a mismatch between source text and the exported amount. It lets you inspect whether the wrong value was selected or the selected text was converted incorrectly.

How Do You Collect Comparable Amazon Prices?
Compare offers with their variation, seller, condition, and delivery destination attached. These fields define which observations can enter the same comparison.
During discovery, collect candidate products from search results, then confirm the selected item in a product detail record. Resolve size and color before taking its price. Store search position and sponsored labels separately if you need them for visibility research.
For page extraction, locate the chosen offer before selecting an amount. For example, put a price stated per 100 ml in the field for unit price. It should not become the bottle’s item price. An installment belongs with its payment schedule and terms. Decide which amount you need from the selected offer, then extract that one; for API data, make the choice from the documented meaning of each price field.
Suppose a result includes a lower amount for a subscription. Save the subscription requirement beside that amount. Record any membership restriction or required buyer action beside the discounted amount.
Store the example USD 1,299.00 as amount 1299.00 and currency USD, keeping the display text for comparison. An absent shipping figure stays unknown, with the absence recorded. Reserve zero for a charge the source actually reports as zero.
Finally, decide which price your report compares. Item price alone may answer one question. Item price plus known shipping answers another. Taxes and conditional benefits need their own treatment; do not label a partial calculation as the final checkout total.
Write these comparison rules before setting alerts. A change of seller starts a separate offer history when the report tracks individual sellers. If the report instead follows the offer selected on each run, retain the seller alongside each observation so the change remains visible.
How Do You Scrape Amazon Reviews Without Overstating Coverage?
Describe the review sample with the data you export. The marketplace, variation filter, language, star selection, and sort order belong in that description. Use the retrieved written reviews for the count, since Amazon’s rating total can include feedback without text.
Use one record per review, with the body, title, rating, date, and variation label kept together. Add the translation status to that record. This keeps the wording being analyzed tied to the relevant comment and product version.
Pagination and filters can show the same review again. Within each marketplace, use its ID to recognize it and count it once, keeping both source references. If there is no stable ID, document your fallback rule and inspect uncertain matches. Two reviewers can write similar sentences.
Continue through the next-page link or token supplied by the source, and log where the run ends. Record a missing next link, repeated page, sign-in screen, or the agreed collection limit as the stopping reason. That information matters when someone later asks how much feedback was collected.
Report the number of unique reviews alongside the filters used. If you selected low-star comments to study defects, describe the issues found in that selection. A claim about how often all buyers encounter a defect needs evidence that the sample represents those buyers.
How Should Your Amazon Scraper Handle Failures?
Treat wrong products, missing values, and blocked requests as separate outcomes. A retry can help only some of them.
Start with the response body: fields present there can be read without rendering the page in a browser. Use a browser when the relevant permitted content requires rendering or interaction. A browser adds work and does not guarantee access. For the client and library choices, see the Python Web Scraping guide.
Develop extraction rules against responses you are permitted to retain. Suppose the saved bottle page has the right item price, but the export contains its price per 100 ml. The error is in field selection. Adjust the selector and use the retained response to verify which price it now reads. A fresh download would introduce another variable: the listing may have changed meanwhile.
A sign-in response delivered with HTTP 200 belongs in the access log and adds no price observation. There is no offer in that response for the product parser to recover.
A rate-limit response calls for following the source’s waiting instructions and deciding whether another attempt is allowed. A selector error calls for correcting the extraction rule. Give retries a finite budget so the job cannot keep cycling through a failure.
The product match is a separate decision. A different variation goes outside that product’s comparison group. For the correct item with no price, retain the product details and mark the price observation incomplete. Availability remains a separate field to inspect.
Count the newly seen review IDs on each page. When a page stops advancing the collection, flag it for inspection rather than letting repeated content inflate the review total.
Route incomplete observations to a review queue before they reach pricing rules. Monitor accepted records, missing required fields, duplicate reviews, and changed page structures. A high request-success rate can coexist with a poor dataset.
When Do Proxies Help With Amazon Data Collection?
Use a proxy when the collection plan calls for a chosen network location or control over exit changes. Define that requirement first: for example, several observations made through a US exit while comparing offers for one delivery destination.
Check the destination returned with the page as well. The client session may retain a delivery choice made earlier, even when the request now uses a US exit. Use the destination shown in the response as evidence of that setting; proxy geography alone cannot supply it.
For a sequence that depends on location selections, keep the client state and proxy allocation together. The client carries cookies and other session state that can affect the settings returned by the site; a sticky session concerns the network exit. If either changes during a run, check that the following observations still use the intended context.
Choose the boundary between independent jobs before enabling rotation. Independent lookups can be handled as separate network sessions. For a page sequence that reuses earlier selections, keep the required state together and investigate an access error before deciding whether a new exit is appropriate.
See the guide to proxy infrastructure for web scraping for routing choices beyond this example. To compare costs, work out what the run cost and how many valid observations it delivered. Include traffic spent on retries and discarded results. Divide the total bill by accepted observations to get the cost of a usable record.
Practical Examples
The following examples use invented figures. They describe no live collection or proxy test.
Example 1: Shipping absorbs the price drop
These quotes are for the same new 500 ml bottle from one seller, with the same delivery destination:
| Field | Observation A | Observation B |
|---|---|---|
| Time | 09:00 UTC | 12:00 UTC |
| Item price (USD) | 29.99 | 24.99 |
| Shipping (USD) | 0.00 | 5.00 |
| Item + shipping (USD) | 29.99 | 29.99 |
Shipping uses up the entire USD 5.00 item-price reduction. The two recorded totals remain USD 29.99. Tax and any unverified promotional benefit have not been included.
If Observation B instead refers to the 250 ml version, place it in that version’s comparison history. The 500 ml history has no matching second quote in this example.
Example 2: An honest review collection summary
Suppose an authorized source displays 2,400 ratings. The first retrieved page contains 12 written reviews. The next contains six review records, two of which repeat the first page. A further request returns a sign-in screen.
The collection summary is:
- Displayed ratings: 2,400.
- Review records retrieved: 18.
- Duplicate records: 2.
- Unique written reviews retained: 16.
- Stop reason: Sign-in required for the next requested page.
- Coverage: Only the retrieved pages; no confirmed count of all written reviews.
Count comments about a chosen issue against the 16 retrieved reviews. That gives a share of this sample. The run has not established how representative those reviews are of all buyers.
Validate a Small Run First
Trace a few exported records back to the source. Resolve unexpected variation changes, missing amounts, and unexplained review gaps before increasing the product list.
If that workflow requires geographic targeting and session control, MangoProxy Residential proxies offer targeting together with rotation and sticky-session options. Check the current product options against your collection requirements, then evaluate them on your authorized workload. Those network features do not establish Amazon access or data completeness.
Glossary
ASIN (Amazon Standard Identification Number): catalog ID associated with the item or selected variation.
API (Application programming interface): the operations and response rules a data source exposes.
SP-API (Amazon Selling Partner API): the API family for authorized seller and vendor applications.
PA-API (Product Advertising API): the legacy version-5 interface referenced by the migration notice.
HTML (HyperText Markup Language): the page markup processed by an extractor.
HTTP (Hypertext Transfer Protocol): the request-and-response protocol used by clients and servers.
JSON (JavaScript Object Notation): structured text for data values, records, and lists.
CSV (Comma-separated values): text rows and columns for spreadsheet import.
IP address (Internet Protocol address): the source or destination address for IP traffic.
Selector: a rule locating a page element for extraction.
Offer snapshot: offer fields saved with their capture time.
Deduplication: handling repeated records so one review is counted once.
Sticky session: a proxy assignment intended to retain an exit for a limited period, subject to availability and the provider’s rules.
Rendering: browser processing that produces the displayed page, including script-driven changes.
UTC (Coordinated Universal Time): A shared time reference for collection timestamps.
Final Thoughts
Send the collection with the context needed to use it: the offer behind each price and the retrieved sample behind each review finding. A later analysis can then be checked against what was actually collected.
Frequently asked questions
Here we answered the most frequently asked questions.
Can I export Amazon product data to Excel?
Yes, when the source permits that use. A CSV with one row per offer observation is convenient for comparing prices in Excel. Give written reviews a separate table and link them using marketplace and ASIN. Import the ASIN column as text so its identifier formatting is retained.
Can scraped product data tell me exactly how many units an item sold?
Not from these fields alone. Use sales records for an exact unit count. A figure inferred from price, ranking, or review activity is an estimate, and the report should identify both the model and the inputs used.
How often should I refresh the data?
A price alert and a periodic review study have different timing requirements. Decide the maximum useful age of the data, check the source’s update schedule and access limits, and try that interval on a small sample. Where supplied, retain the source’s refresh timestamp alongside your collection timestamp.