SW for extracting data from graph

Hi, I’m looking for some way to extract data from an interactive graph, so that I can do calculations on it. Does anyone know of suitable software, or have any idea how to write this?

Here’s an example of what I’m thinking about:

The app would have to drag the cursor over the graph, log the data to separate columns in a file, then select a new week and repeat.

This is a web scraping task. How to capture data from a web page.

As far as I know, you’ve pointed at an example site which is impossible to scrape.

Scraping involves looking at the web page source, finding the document elements of interest, and grabbing them. This page has none of that: the data is on the server, rendered on the server, and presented using javascript.

If you can come up with a way to:

  1. make a screen grab of the web page presentation
  2. extract the data from the screen grab (i.e. OCR from an image)

then I think you’ll get somewhere.

This should be pretty easy. Just make HTTP requests to the same API that the chart is making requests to. You can figure out which URL to make HTTP requests to by opening the developers panel in your browser and navigating to the networks tab. For example, I believe the chart starts by showing data from here. This approach is much easier than scraping the webpage.

1 Like