IPFScrape

Learn how to configure IPFScrape for use with Filebase.

What is IPFScrape?

IPFScrape is a CLI tool that downloads all of the contents of a webpage and uploads it to IPFS. Once uploaded, it needs to be pinned to IPFS for long-term storage. IPFScrape can be used to archive websites, or create copies that can be edited and shared using the IPFS CID.

Read below to learn how to use IPFScrape with Filebase.

Prerequisites:

• Download and run the IPFS daemon.

• Sign up for a free Filebase account.

• Have your Filebase Access and Secret Keys. Learn how to view your access keys here.

• Create a Filebase IPFS Bucket. Learn how to create a bucket here.

1. Start by cloning the tool’s repository:

git clone https://github.com/victorb/ipfscrape

2. Then, navigate into the directory and run the run.sh script. Replace $URL with the URL of the webpage you’d like to scrape.

cd ipfscrape

./run.sh $URL

The script will return a URL containing the scraped file’s CID:

3. Copy the CID value.

4. Then, navigate to the Filebase web console dashboard. Navigate inside an IPFS bucket.

Select ‘Upload’ then select ‘CID’.

5. Enter the CID you copied previously. Name the pin ‘index.html’.

6. Once pinned, you can access the webpage using the CID and the Filebase IPFS gateway:

https://ipfs.filebase.io/ipfs/[CID]