feat: add on disk blob pool

[mattsse]

adds a simple file based on disk blob store

blobs are stored rlp encoded in individual files, file derived from hash

[DoTheBestToGetTheBest]

Hey, this is amazing. I wrote some method, i want to push the code but i can't

impl BlobStore for DiskFileBlobStore {
    fn insert(&self, tx: B256, data: BlobTransactionSidecar) -> Result<(), BlobStoreError> {
        todo!()
    }

    fn insert_all(&self, txs: Vec<(B256, BlobTransactionSidecar)>) -> Result<(), BlobStoreError> {
        todo!()
    }

    fn delete(&self, tx: B256) -> Result<(), BlobStoreError> {
        let file_path = self.inner.blob_dir.join(tx.to_string());
        if file_path.exists() {
            let file_size = file_path.metadata().unwrap().len();
            fs::remove_file(&file_path).unwrap();
            // Decrement the size tracker by the size of the removed file
            self.inner.size_tracker.sub_size(file_size as usize);
            // Decrement the blob count by 1
            self.inner.size_tracker.update_len(self.inner.size_tracker.blobs_len() - 1);
            Ok(())
        } else {
            Err(BlobStoreError::MissingSidecar(tx))
        }
    }

    fn delete_all(&self, txs: Vec<B256>) -> Result<(), BlobStoreError> {
        if txs.is_empty() {
            return Ok(())
        }

        let mut total_sub = 0;
        let mut count_removed = 0;

        for tx in txs {
            let file_path = self.inner.blob_dir.join(tx.to_string());
            if file_path.exists() {
                let file_size = file_path.metadata().unwrap().len() as usize;
                fs::remove_file(&file_path).unwrap();
                total_sub += file_size;
                count_removed += 1;
            }
        }

        self.inner.size_tracker.sub_size(total_sub);
        self.inner.size_tracker.update_len(self.inner.size_tracker.blobs_len() - count_removed);
        Ok(())
    }

    fn get(&self, tx: B256) -> Result<Option<BlobTransactionSidecar>, BlobStoreError> {
        let file_path = self.inner.blob_dir.join(tx.to_string());
        if file_path.exists() {
            let mut data = fs::read(&file_path).unwrap();
            let sidecar = BlobTransactionSidecar::decode_inner(&mut data.as_slice())
                .map_err(|e| BlobStoreError::DecodeError(e))
                .unwrap();

            Ok(Some(sidecar))
        } else {
            Ok(None)
        }
    }

    fn get_all(
        &self,
        txs: Vec<B256>,
    ) -> Result<Vec<(B256, BlobTransactionSidecar)>, BlobStoreError> {
        let mut items = Vec::with_capacity(txs.len());

        for tx in txs {
            let file_path = self.inner.blob_dir.join(tx.to_string());
            if file_path.exists() {
                let mut data = fs::read(&file_path).unwrap();
                let sidecar = BlobTransactionSidecar::decode_inner(&mut data.as_slice())
                    .map_err(|e| BlobStoreError::DecodeError(e))?;

                items.push((tx, sidecar));
            }
        }

        Ok(items)
    }

    fn get_exact(&self, txs: Vec<B256>) -> Result<Vec<BlobTransactionSidecar>, BlobStoreError> {
        let mut items = Vec::with_capacity(txs.len());

        for tx in txs {
            let file_path = self.inner.blob_dir.join(tx.to_string());
            if file_path.exists() {
                let mut data = fs::read(&file_path).unwrap();
                let sidecar = BlobTransactionSidecar::decode_inner(&mut data.as_slice())
                    .map_err(|e| BlobStoreError::DecodeError(e))
                    .unwrap();

                items.push(sidecar);
            } else {
                return Err(BlobStoreError::MissingSidecar(tx))
            }
        }

        Ok(items)
    }

    fn data_size_hint(&self) -> Option<usize> {
        Some(self.inner.size_tracker.data_size())
    }

    fn blobs_len(&self) -> usize {
        self.inner.size_tracker.blobs_len()
    }
}

[DoTheBestToGetTheBest]

I didn't know how to write the insert since i can't convert data to bytes so if any hint on this i can continue for it

[DoTheBestToGetTheBest]

Changed also decode and encode method to public since it was private method

[mattsse]

amazing, do you want to branch off this PR, make changes and then open a new PR against this branch, so I can review?

[DoTheBestToGetTheBest]

amazing, do you want to branch off this PR, make changes and then open a new PR against this branch, so I can review?

it is not better to just push to your branch ? what ever you want, i do it, no problem for me boss :)

[Rjected]

LGTM so far, pretty simple and should work

[Rjected]

guessing this part will be implemented later because this part of the config is ignored when the store is created

[mattsse]

yeah, since we don't retire pooled txs to disk, this is redundant atm, just a placeholder

[DaniPopes]

LGTM