> For the complete documentation index, see [llms.txt](https://openup.gitbook.io/training-content/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://openup.gitbook.io/training-content/the-fundamentals-of-data-driven-storytelling/module-2-get/2.2-an-introduction-to-spreadsheet-software/simple-web-scraping-with-google-sheets.md).

# Simple web scraping with Google Sheets

Our goal, as discussed, is to get data into a spreadsheet application which can be used to perform analysis. In the last topic, we looked at scraping data from PDFs with **Tabula**.

If a table of data has been published on a web page, sometimes you can scrape it directly into your spreadsheet application. Here we are going to look at an example with **Google Sheets**.&#x20;

Take a look at this page, which shows current and historical tender notices for the state owned entity (SOE) **CIPC**.  <http://www.cipc.co.za/index.php/tenders-and-supplier-database-form/>

<figure><img src="https://2315907434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi8JLXAtCzbzpRcnAIJsv%2Fuploads%2FSUJsjCsdOV6aUFFE8bmH%2Fspaces_uSSbOeqjFMxbM6oPQFVO_uploads_2JcPlQUMl9Pn1dEd5bJw_image.webp?alt=media&amp;token=474e120e-3d67-4e88-a1f3-6fa735f5ffdf" alt=""><figcaption></figcaption></figure>

It doesn't look like it, but the list of tenders on this page is published as a table. If we want to import this list - for example to count how many tenders were issued in different months of the year - we can do it directly from within Google Sheets. First of all, we open up a new spreadsheet.

<figure><img src="https://2315907434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi8JLXAtCzbzpRcnAIJsv%2Fuploads%2FCHPxe00OcCczKWBHmZju%2Fspaces_uSSbOeqjFMxbM6oPQFVO_uploads_c1YTECJHB7FFXvs77qXA_image.webp?alt=media&amp;token=d9d0ec25-2ddc-43f9-bb14-09c044d0edac" alt=""><figcaption></figcaption></figure>

&#x20;Put your cursor in cell A1 (for more information about how to navigate spreadsheets, see our **Data Pipeline** module). Now type the following formula. **=IMPORTHTML("<http://www.cipc.co.za/index.php/tenders-and-supplier-database-form/>", "table", 1)**

Let's pause for a second to look at that formula.

The equals sign at the start tells Google Sheets that this cell is a formula, not a string of text and numbers (like a paragraph). A formula is an instruction to do something, for example **=sum(A2:E2)** tells Google Sheets to add the values in column 2, rows A to E together. **=average(A2:E2)** would look for the arithmetic mean of the same numbers.&#x20;

In our formula, **=importhtml** tells Google Sheets to load the webpage inside the brackets **("<http://www.cipc.co.za/index.php/tenders-and-supplier-database-form/>")**. It's important to notice the quote marks around the website URL.

Each element of the formula is separate by commas. The second element tells Google Sheets to load a **table** from the webpage, and the final element - the **1** - which table to load. For humans, reading the formula backwards makes more sense:&#x20;

<mark style="color:green;">"Load the first table you find on</mark> <mark style="color:green;"></mark><mark style="color:green;">**<http://www.cipc.co.za/index.php/tenders-and-supplier-database-form>**</mark><mark style="color:green;">. It's in HTML"</mark>

### Working with imported data

After you've entered the formula, your spreadsheet should look like this.&#x20;

<figure><img src="https://2315907434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi8JLXAtCzbzpRcnAIJsv%2Fuploads%2FxFjWYQgJeIwJUz28CbMR%2Fspaces_uSSbOeqjFMxbM6oPQFVO_uploads_n0WRLEfmp0BIunxfQL9V_image.webp?alt=media&amp;token=3d387630-bab1-4920-803b-1a1da4b6521b" alt=""><figcaption></figcaption></figure>

We can tidy this up by making the columns a little wider. Just put your cursor on the line between A and B at the top of the sheet, for example, and drag it manually to the right. You can also double click this line to automatically resize the column.&#x20;

<figure><img src="https://2315907434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi8JLXAtCzbzpRcnAIJsv%2Fuploads%2FwOBBhmn9OoKGCwmCy9rr%2Fspaces_uSSbOeqjFMxbM6oPQFVO_uploads_vWhycwwDpRWa9iSbF1Wy_image.webp?alt=media&amp;token=be552ce8-2937-45a5-a2b8-a4150016d23b" alt=""><figcaption></figcaption></figure>

Now we have a table of all the latest tenders from CIPC, in a machine readable format. Because this table is reloaded from the original website every time we refresh our sheet, you might want to copy the data onto a fresh tab to store a copy for working on. Just press **CTRL + A** to select the whole table,  and click the **+** icon in the bottom left to add a new tab to your spreadsheet. In Cell A1 on the new sheet, press **CTRL + SHIFT + V**. This is the command to **paste values only**, in other words, it takes the characters in each cell and not the formula.

<figure><img src="https://2315907434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi8JLXAtCzbzpRcnAIJsv%2Fuploads%2Fx9Gou9tZuxpw01hVjUMi%2Fspaces_uSSbOeqjFMxbM6oPQFVO_uploads_BQQ4HTT1kk68WlPlUrWg_image.webp?alt=media&amp;token=6f230d2b-e776-4ecb-8751-cb2c88fa47ce" alt=""><figcaption></figcaption></figure>

Now your spreadsheet is ready for work.
