IPFS Pinning
Learn more about IPFS pinning and how to pin data to IPFS

What is IPFS Pinning?

IPFS pinning refers to the process of specifying data to be retained and persist on one or more IPFS nodes. Pinning assures that data is accessible indefinitely, and will not be removed during the IPFS garbage collection process.

Understanding the Garbage Collection Process

When files and data are stored on the IPFS network, nodes on the network cache the files that they download and keep those files available for other nodes on the network. Since storage on these nodes is finite, the cache for each node must be cleared periodically to make room for new files to be cached and made available. The process of clearing the cache for IPFS nodes is referred to as the IPFS garbage collection process.
Garbage collection is an automatic process that is used to manage resources, such as IPFS node disk space. The process is designed to remove cached data that it thinks is no longer needed, though if your IPFS CID refers to a file that is vital to your workflow or environment, having this file removed can be detrimental. So how do you prevent your file from being removed during this process?

IPFS Pinning

To protect data from the garbage collection process, data must be pinned on the IPFS network. This ensures that data is retained indefinitely and is always accessible. Pinning is useful for a variety of workflows, such as accessing data files from around the world without managing sharing permissions. All data uploaded to IPFS is public by default since all you need to access it is the file’s CID. There are no permissions, user accounts, or other security settings tied to IPFS CIDs.
One of the most popular workflows utilizing IPFS pinning right now are NFT collections. These collections have a variety of files involved, including NFT image files and their associated metadata files. If these files aren’t pinned on IPFS and they get removed by the IPFS garbage collection process, this can result in an “NFT Rug Pull”, which means that the NFT ceases to exist, and is no longer accessible or transferable.

Pinning Services

IPFS pinning can be configured on locally hosted IPFS nodes, but for external, long-term storage, that’s where pinning services such as Filebase come in.

How To Pin Data Onto IPFS With Filebase

Files uploaded to an IPFS on Filebase are automatically pinned to IPFS and stored with 3x replication across the Filebase infrastructure by default, at no extra cost to you. This means your data is accessible and reliable in the event of a disaster or outage, and won't be affected by the IPFS garbage collection process.

Uploading a File to IPFS Through The Filebase Web Dashboard

1. Start by clicking on the ‘Buckets’ option from the menu to open the Buckets dashboard.
2. Select your IPFS Bucket.
3. After clicking on the bucket name, you will see any previously uploaded files. To upload another file, select 'Upload', then select 'File' from the options.
4. Select the file you want to upload to the IPFS.
5. Once uploaded, you will be able to view and copy the IPFS CID from the 'CID' category, as seen below.

Uploading a Folder to IPFS Through The Filebase Web Console

1. Start by clicking on the ‘Buckets’ option from the menu to open the Buckets dashboard.
2. Select your IPFS Bucket.
3. After clicking on the bucket name, you will see any previously uploaded files. To upload another file, select 'Upload', then select 'Folder' from the options.
4. Select the folder you'd like to upload to IPFS.
5. Once uploaded, the folder will look similar to IPFS individual files.
6. Copy the IPFS CID for your folder, then navigate to https://ipfs.filebase.io/ipfs/[CID]. The contents of your folder will be listed.

Uploading a File to IPFS Using the S3-Compatible API

If you're using the S3 compatible API, the CID will be returned in the response of a PutObject call.
For example, if we run the following AWS CLI command:
aws --endpoint https://s3.filebase.com s3 cp test-images/7FIMFhlMf6A.jpg s3://ipfs-test --debug
The response is shown below. For convenience, we've highlighted the respective response header:
You can also call the HeadObject API to fetch the CID at any time as well:
aws --endpoint https://s3.filebase.com s3api head-object --bucket ipfs-test --key 7FIMFhlMf6A.jpg

Uploading a Folder to IPFS Through the S3-Compatible API

A CAR file is a type of compressed archive file that contains multiple files, similar to a ZIP file. CAR files are used by the FileCoin and IPFS networks, which utilize the metadata fields to include the IPFS CIDs of each file within the archive.
Filebase supports uploading CAR files to IPFS using the S3-compatible API through the PutObject or MultipartUpload methods.
When you upload a CAR file, you can pass an S3 metadata header of x-amz-meta-import: car along with the request. Filebase will then import the file as a CAR, and return the resulting CID.
Below is an example using the AWS CLI, which adds the x-amz-meta prefix automatically:
aws --endpoint https://s3.filebase.com s3 cp pictures.car s3://ipfs-test --debug --metadata 'import=car'
With the specified –debug flag, the response headers are shown, which reveal the CID:
2022-06-14 19:51:41,400 - ThreadPoolExecutor-0_2 - botocore.parsers - DEBUG - Response headers: {'Date': 'Tue, 14 Jun 2022 23:51:41 GMT', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'x-amz-meta-cid': 'bafybeieurpeyzighqrvwjqyj3szuvbqvrbijm7cdair5a422ipf2d5qnlq', 'ETag': 'W/"d8cad258a3d9bbe03a26a13a3ec43b21"', 'x-amz-request-id': '144e0415-8162-45cd-b071-e51dada956ae', 'x-amz-id-2': 'ZmlsZWJhc2UtNmQ3ZjQ5OGZmZC14ejk1Mg==', 'Server': 'Filebase'}

Retrieving IPFS CIDs Using The AWS S3 SDK for JavaScript

You can also retrieve the IPFS CID using different AWS S3 SDKs. The example below uses The JavaScript SDK:
1
const AWS = require('aws-sdk');
2
const fs = require('fs');
3
4
const s3 = new AWS.S3({
5
endpoint: 'https://s3.filebase.com',
6
region: 'us-east-1',
7
signatureVersion: 'v4',
8
});
9
10
fs.readFile('image.png', (err, data) => {
11
if (err) {
12
console.error(err);
13
return;
14
}
15
16
const params = {
17
Bucket: 'my-ipfs-bucket',
18
Key: 'test/image.png',
19
ContentType: 'image/png',
20
Body: data,
21
};
22
23
const request = s3.putObject(params);
24
request.on('httpHeaders', (statusCode, headers) => {
25
console.log(`CID: ${headers['x-amz-meta-cid']}`);
26
});
27
request.send();
28
});
Copied!
If you have any questions, please join our Discord server, or send us an email at [email protected]
Last modified 10d ago