Skip to content

Commit

Permalink
Fix problem with append datapoint with same date and price as latest …
Browse files Browse the repository at this point in the history
…datapoint

Can now scrape multiple times the same day without append the same datapoint, now just updates the price in the lastest datapoint if the lastest datapoint's date is the same date as today
  • Loading branch information
Crinibus committed Apr 13, 2021
1 parent 6072d11 commit 812ccda
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion scraper/scrape.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,12 @@ def update_data(self) -> dict:
product_info["info"].update(
{"url": short_url, "id": product_id, "currency": self.info.currency}
)
product_info["datapoints"].append({"date": date, "price": self.info.price})

latest_datapoint = product_info["datapoints"][-1]

if latest_datapoint["date"] == date:
latest_datapoint["price"] = self.info.price
else:
product_info["datapoints"].append({"date": date, "price": self.info.price})

return data

0 comments on commit 812ccda

Please sign in to comment.