IPFS
Learn more about Filebase’s integration with the IPFS network.

What is IPFS?

InterPlanetary File System, or IPFS, is a distributed and decentralized storage network for storing and accessing files, websites, data, and applications. IPFS uses peer-to-peer network technology to connect a series of nodes located across the world that make up the IPFS network.

How does IPFS work?

IPFS is unique from other decentralized storage networks because it offers additional features and attributes such as content addressing, directed acyclic graphics (DAGs) and distributed hash tables (DHTs).

Unique Data Identification via Content Addressing

Data stored on IPFS is located through it’s content address rather than it’s physical location. When data is stored on IPFS, it is stored in a series of encrypted pieces, with each piece having its own unique content identifier, or hash. This hash serves as an identifier and links the piece to all the other pieces of that data.
Identifying an object, such as an object or a node, by the value of its hash is referred to as content addressing. The hash identifier is known as the Content Identifier, or CID. When objects are uploaded to an IPFS bucket on Filebase, the IPFS CID is listed in the object’s metadata for easy reference in any tool or application, or for use with an IPFS gateway.
Learn more about IPFS CIDs in our deep dive document about CIDs below:

Content Linking via Directed Acyclic Graphs (DAGs)

Directed Acyclic Graphs are a hierarchical data structure. A graph is a way to display objects and the relationship between them. A directed graph is when a graph’s edges have direction, as depicted in the photo above. An acyclic graph is a graph where the edges have definitive ends and do not create a loop to other objects. Think of a family tree that shows ancestors and their relationship to one another. This is a good example of a Directed Acyclic Graph.
In this context, an object in a graph is referred to as a node and an edge refers to the relation between the objects in a graph.
IPFS uses Merkle DAGS, where each node has a unique identifier that is the result of hashing the node’s contents. Merkle DAGs are a form of self-verified data structures.

Content Discovery through Distributed Hash Tables (DHTs)

A distributed hash table (DHT) is a distributed system for mapping keys to their associated values. DHTs are databases of keys and values that are split across all the peers on a distributed network. To locate content, you ask a peer on the network, which will return a DHT that tells you which peers are storing which blocks of content that make up the data object you’re requesting.

Native IPFS URLs

Applications that natively support IPFS content addressing can refer to content stored on IPFS in the format:
ipfs://{CID}/{optional path to resource}
This format doesn’t work for applications or tools that rely on HTTP, such as Curl or Wget. For these tools, you need to use an IPFS gateway.

IPFS Gateways

Content stored on IPFS can be accessed by using an IPFS gateway. Gateways are used to provide workarounds for applications that don’t natively support IPFS. An IPFS gateway can be local, private, or public, and uses the IPFS content ID to provide a URL link to the content for access to the stored content.
Filebase's native IPFS gateway is as follows: https://ipfs.filebase.io/ipfs/{CID}
The Filebase Public IPFS Gateway has an effective rate limit of 200 RPM (requests per minute).
All content stored on IPFS through Filebase can be accessed through the Filebase gateway with faster response times than accessing the content through any other gateway. This is because the Filebase gateway is peered with our IPFS nodes. The Filebase gateway is also peered with the IPFS gateways of other pinning services.

IPFS Subdomain Gateways

The format of a subdomain gateway is as follows:
https://{CID}.ipfs.dweb.link
Using a subdomain gateway as a drop-in replacement for a traditional gateway removes the need for there to be a CID version translation because there is a difference between opening an older CIDv0 resource and a CIDv1 resource. When a CIDv0 is accessed with a traditional gateway, the domain will return an HTTP 301 redirect to a subdomain, converting the CIDv0 CID to CIDv1 CID.
For example, opening a CIDv0 resource using a traditional gateway at:
https://dweb.link/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
returns a redirect to a CIDv1 representation at:
https://bafkreidgvpkjawlxz6sffxzwgooowe5yt7i6wsyg236mfoks77nywkptdq.ipfs.dweb.link/
By using a subdomain gateway initially, there is no need for the conversion step to take place.
For more information on IPFS CIDs and fetching IPFS links, see our guide here.

What is ‘pinning’ data with IPFS?

When data is stored in a Filebase bucket, Filebase’s native edge caching technology keeps that data cached at the network’s edge locations for quick access to that data again. Then, when the IPFS garbage collection storage limit is reached, which varies between the IPFS Desktop client and Brave, the cached data is cleared to make room for other data. This clearing of unpinned data is referred to as the IPFS garbage collection process.
If you want data to be cached for a long period of time you can pin the data. This will keep the data cached on the IPFS network indefinitely, since pinned data is skipped during the garbage collection process, allowing for faster data retrieval times.
For more detailed information on IPFS pinning, see our deep dive below:
Files uploaded to an IPFS on Filebase are automatically pinned to IPFS and stored with 3x replication across the Filebase infrastructure by default, at no extra cost to you. This means your data is accessible and reliable in the event of a disaster or outage, and won't be affected by the IPFS garbage collection process.
Pinning data is a unique feature of the IPFS network, and is not offered for buckets created on other networks.

How do I create an IPFS Bucket on Filebase?

Simply navigate to your Dashboard Console, create a new bucket and choose the IPFS option.

How To Pin Data Onto IPFS With Filebase

Files uploaded to an IPFS on Filebase are automatically pinned to IPFS and stored with 3x replication across the Filebase infrastructure by default, at no extra cost to you. This means your data is accessible and reliable in the event of a disaster or outage, and won't be affected by the IPFS garbage collection process.

Uploading a File to IPFS Through The Filebase Web Dashboard

1. Start by clicking on the ‘Buckets’ option from the menu to open the Buckets dashboard.
2. Select your IPFS Bucket.
3. After clicking on the bucket name, you will see any previously uploaded files. To upload another file, select 'Upload', then select 'File' from the options.
4. Select the file you want to upload to the IPFS.
5. Once uploaded, you will be able to view and copy the IPFS CID from the 'CID' category, as seen below.

Uploading a Folder to IPFS Through The Filebase Web Console

1. Start by clicking on the ‘Buckets’ option from the menu to open the Buckets dashboard.
2. Select your IPFS Bucket.
3. After clicking on the bucket name, you will see any previously uploaded files. To upload another file, select 'Upload', then select 'Folder' from the options.
4. Select the folder you'd like to upload to IPFS.
5. Once uploaded, the folder will look similar to IPFS individual files.
6. Copy the IPFS CID for your folder, then navigate to https://ipfs.filebase.io/ipfs/[CID]. The contents of your folder will be listed.
U

Uploading a File to IPFS Using the S3-Compatible API

If you're using the S3 compatible API, the CID will be returned in the response of a PutObject call.
For example, if we run the following AWS CLI command:
aws --endpoint https://s3.filebase.com s3 cp test-images/7FIMFhlMf6A.jpg s3://ipfs-test --debug
The response is shown below. For convenience, we've highlighted the respective response header:
You can also call the HeadObject API to fetch the CID at any time as well:
aws --endpoint https://s3.filebase.com s3api head-object --bucket ipfs-test --key 7FIMFhlMf6A.jpg

Uploading a Folder to IPFS Through the S3-Compatible API

A CAR file is a type of compressed archive file that contains multiple files, similar to a ZIP file. CAR files are used by the FileCoin and IPFS networks, which utilize the metadata fields to include the IPFS CIDs of each file within the archive.
Filebase supports uploading CAR files to IPFS using the S3-compatible API through the PutObject or MultipartUpload methods.
When you upload a CAR file, you can pass an S3 metadata header of x-amz-meta-import: car along with the request. Filebase will then import the file as a CAR, and return the resulting CID.
Below is an example using the AWS CLI, which adds the x-amz-meta prefix automatically:
aws --endpoint https://s3.filebase.com s3 cp pictures.car s3://ipfs-test --debug --metadata 'import=car'
With the specified –debug flag, the response headers are shown, which reveal the CID:
2022-06-14 19:51:41,400 - ThreadPoolExecutor-0_2 - botocore.parsers - DEBUG - Response headers: {'Date': 'Tue, 14 Jun 2022 23:51:41 GMT', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'x-amz-meta-cid': 'bafybeieurpeyzighqrvwjqyj3szuvbqvrbijm7cdair5a422ipf2d5qnlq', 'ETag': 'W/"d8cad258a3d9bbe03a26a13a3ec43b21"', 'x-amz-request-id': '144e0415-8162-45cd-b071-e51dada956ae', 'x-amz-id-2': 'ZmlsZWJhc2UtNmQ3ZjQ5OGZmZC14ejk1Mg==', 'Server': 'Filebase'}

Retrieving IPFS CIDs Using The AWS S3 SDK for JavaScript

You can also retrieve the IPFS CID using different AWS S3 SDKs. The example below uses The JavaScript SDK:
1
const AWS = require('aws-sdk');
2
const fs = require('fs');
3
4
const s3 = new AWS.S3({
5
endpoint: 'https://s3.filebase.com',
6
region: 'us-east-1',
7
signatureVersion: 'v4',
8
});
9
10
fs.readFile('image.png', (err, data) => {
11
if (err) {
12
console.error(err);
13
return;
14
}
15
16
const params = {
17
Bucket: 'my-ipfs-bucket',
18
Key: 'test/image.png',
19
ContentType: 'image/png',
20
Body: data,
21
};
22
23
const request = s3.putObject(params);
24
request.on('httpHeaders', (statusCode, headers) => {
25
console.log(`CID: ${headers['x-amz-meta-cid']}`);
26
});
27
request.send();
28
});
Copied!
If you have any questions, please join our Discord server, or send us an email at [email protected]