Fetch latest blocks from database

[saimeunt]

Fetch latest blocks from database

⚠️ Reading contributors guidelines to get assigned is MANDATORY!

Read contributors guidelines

User stories

As a rollup developer using op-scan, I want to fetch the latest blocks data either from JSON-RPC (current behavior) or from a database fed from op-indexer.
The data source should be abstracted away in a dedicated function and is configured using an environment variable.

Validation

It should fetch the exact same data whether calling the RPC directly or querying the database.
It MUST adhere to the existing design pattern already implemented in the tx details page.

Implementation

Create a new helper function under components/pages/blocks/fetch-blocks.ts, use the same logic found in components/pages/tx/fetch-transaction-details.ts, a top level exported function which is data source agnostic and 2 separate functions fetching data from either JSON-RPC (this is already implemented here: https://github.com/walnuthq/op-scan/blob/main/src/lib/fetch-data.ts#L20-L28) or from the database (this is what has to be implemented).
Move the fetchLatestBlocks data fetching logic from the lib/fetch-data.ts to this helper function and just call this function in the blocks page top-level server component.
You will need to make sure the pagination logic is preserved when querying data via Prisma.

Run op-indexer as a background task in another terminal along the explorer to keep an up-to-date database mirroring what you'd fetch from the JSON-RPC endpoint.

Resources

• fetch tx details helper function: https://github.com/walnuthq/op-scan/blob/main/src/components/pages/tx/fetch-transaction-details.ts
• data source agnostic function called in tx detail page: https://github.com/walnuthq/op-scan/blob/main/src/components/pages/tx/index.tsx#L10
• op-indexer getting started https://github.com/walnuthq/op-scan/blob/main/README.md#run-the-indexer

[Iwueseiter]

Hi @saimeunt can I work on this?

[onlydustapp]

Hi @Iwueseiter!
Maintainers during the ODHack #6.0 will be tracking applications via OnlyDust.
Therefore, in order for you to have a chance at being assigned to this issue, please apply directly here, or else your application may not be considered.

[Jemiiah]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

hello @saimeunt I would love to work on this issue I'm a frontend developer and I have made contribution to other repositories, and this would be my first contribution on this repository I would love to contribute to this project!!

How I plan on tackling this issue

Here is what is required to be done:
Create a New Helper Function:
Develop a new helper function under the specified path: components/pages/blocks/fetch-blocks.ts.
Construct this function to abstract the data source and use the same logic as the fetch-transaction-details.ts found in components/pages/tx.
The new function should be designed to fetch data from either the JSON-RPC endpoint or the database, depending on the environment variable configuration.

Transfer Fetch Logic:
Move the fetchLatestBlocks data-fetching logic from the existing lib/fetch-data.ts to the newly created helper function.

Top-Level Server Component:
Integrate the newly created fetchBlocks function into the top-level server component on the blocks page.

Database Integration:
Create a function to fetch blocks data from the database using Prisma, ensuring that the pagination logic is preserved when querying data via Prisma.

Data Consistency Verification:
Ensure that the data fetched from both the JSON-RPC endpoint and the database is consistent by verifying that the structures match and the data reflects the same blocks.

Op-Indexer Utilization:
Run op-indexer as a background task in a separate terminal alongside the explorer to maintain an up-to-date database, mirroring what would be fetched from the JSON-RPC endpoint.

[Iwueseiter]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

I'm a blockchain developer and also a frontend developer. I've contributed to projects here on onlydust before.

How I plan on tackling this issue

Here is how I intend to work on this task: Define an environment variable to toggle between JSON-RPC and database data sources.
Then I'd Create a new helper function in components/pages/blocks/fetch-blocks.ts. This function will be responsible for fetching the blocks data from the appropriate source.
Create a function to fetch blocks data from the database. then I'd Move the existing fetch logic from lib/fetch-data.ts to the new helper function. Adjust any import paths as necessary.
Use the new fetchBlocks function in the top-level server component on the blocks page. I'd also ensure that the data fetched from both JSON-RPC and the database is consistent by verifying that the structures match and the data reflects the same blocks. I'd ensure that op-indexer is running in a separate terminal window to keep the database updataed.
I'd tests to verify that both data sources return identical data. lastly, Implement error handling to manage connectivity issues with JSON-RPC or the database.

[martinvibes]

hello @saimeunt I would love to be assigned this issue and be a contributor on this project hopefully

My background and how it can be leveraged
i'm a frontend developer and i have experience using html, css, tailwind css, react, JavaScript and a little of TypeScript. etc.

Here is my Onlydust profile link https://app.onlydust.com/u/martinvibes. Please give me the opportunity.

How I plan on tackling this issue

To solve this issue I’ll:
Create a new helper function in components/pages/blocks/fetch-blocks.ts. Move the fetchLatestBlocks data fetching logic from lib/fetch-data.ts to this helper function. I’ll ensure that the new function is data source agnostic by abstracting the data source using environment variables. I’ll be careful to preserve pagination logic when querying data via Prisma. I’ll also run op-indexer as a background task to keep the database up-to-date.

[onlydustapp]

Hi @martinvibes!
Maintainers during the ODHack #6.0 will be tracking applications via OnlyDust.
Therefore, in order for you to have a chance at being assigned to this issue, please apply directly here, or else your application may not be considered.

[ShantelPeters]

I am applying to this issue via https://app.onlydust.com/p/opscan-by-walnut

@saimeunt hello ,I am a frontend developer and this would be my first time contributing to this ecosystem

[od-hunter]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

Please I will like to be assigned this issue. I am a front-end Developer, and my experience includes html, css, react, JavaScript and a little of TypeScript.
Here is my Onlydust profile link https://app.onlydust.com/u/od-hunter . Please give me the opportunity.

How I plan on tackling this issue

To solve this issue I’ll:
Create a new helper function in components/pages/blocks/fetch-blocks.ts. Move the fetchLatestBlocks data fetching logic from lib/fetch-data.ts to this helper function. I’ll ensure that the new function is data source agnostic by abstracting the data source using environment variables. I’ll be careful to preserve pagination logic when querying data via Prisma. I’ll also run op-indexer as a background task to keep the database up-to-date.

[blessingbytes]

Hi @saimeunt please I would love to be assigned this issue and be a contributor to your repo

[onlydustapp]

Hi @blessingbytes!
Maintainers during the ODHack #6.0 will be tracking applications via OnlyDust.
Therefore, in order for you to have a chance at being assigned to this issue, please apply directly here, or else your application may not be considered.

[Ugo-X]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

I am a Full Stack blockchain Developer with expertise in Next.js, Nest.js, TypeScript, JavaScript, React, Node.js, Three.js, and Solidity. My journey with OnlyDust hackathons began at Edition 1, and I've since made 30 contributions across 9 projects. With my extensive experience on the OnlyDust platform (profile: https://app.onlydust.com/u/Ugo-X), I've honed my skills in delivering quality solutions under pressure.
I bring a unique blend of technical proficiency and user-focused design to every project, whether it's crafting immersive 3D experiences or developing smart contracts. My track record shows I can adapt quickly and contribute effectively to diverse challenges.
As we surf through Edition 6, I'm excited to leverage my skills and hackathon experience to push the boundaries of blockchain development. I'm confident in my ability to tackle new challenges and drive innovation in this space.

How I plan on tackling this issue

I will...

1. Thoroughly read and understand the contributors' guidelines:

   • Carefully review all documentation related to contributing to the project
   • Take note of coding standards, commit message formats, and pull request procedures

2. Set up the development environment:

   • Clone the op-scan repository
   • Install all necessary dependencies
   • Set up the database and ensure it's properly configured
   • Run op-indexer as a background task to keep the database updated

3. Create a new helper function:

   • Create a new file: src/components/pages/blocks/fetch-blocks.ts
   • Implement a top-level exported function that is data source agnostic
   • Use the same logic pattern as found in fetch-transaction-details.ts

4. Implement JSON-RPC fetching:

   • Move the existing fetchLatestBlocks function from lib/fetch-data.ts to the new file
   • Refactor the function to work within the new structure
   • Ensure all pagination logic is preserved

5. Implement database fetching:

   • Create a new function to fetch blocks data from the database using Prisma
   • Structure the Prisma query to match the pagination used in RPC calls
   • Ensure the returned data structure mirrors that of the JSON-RPC method

6. Implement data source abstraction:

   • Add logic to determine the data source based on an environment variable
   • Create a switch statement or if-else logic to call the appropriate fetching function

7. Update the blocks page component:

   • Modify the blocks page top-level server component to use the new fetch-blocks function
   • Remove any direct calls to the old fetchLatestBlocks function
   • Ensure the component works with both data sources

8. Set up environment variable:

   • Add a new environment variable (e.g., BLOCKS_DATA_SOURCE) to .env file
   • Update the fetch-blocks function to use this variable for source selection
   • Document the usage of this new environment variable

9. Implement error handling and logging:

   • Add try-catch blocks to handle potential errors in data fetching
   • Implement appropriate error messages for different failure scenarios
   • Add logging to track the data source being used and any issues that arise

10. Ensure data consistency:

    • Verify that the data structure is identical whether fetched from JSON-RPC or database
    • Create a function to compare the results from both sources for testing purposes

11. Preserve pagination logic:

    • Carefully implement pagination in the Prisma query to match the RPC method
    • Test pagination thoroughly with both data sources

12. Write unit tests:

    • Create test cases for both RPC and database fetching
    • Test pagination logic thoroughly
    • Ensure data consistency between the two sources

13. Perform integration testing:

    • Test the entire flow from the UI to the database
    • Verify that switching between data sources works as expected
    • Ensure that the UI behaves identically regardless of the data source

14. Optimize performance:

    • Profile the database queries and optimize if necessary
    • Consider implementing caching mechanisms if appropriate

15. Update documentation:

    • Add comments to the new fetch-blocks function explaining its usage
    • Update the project README with information about the new feature
    • Create or update API documentation if necessary

16. Prepare the pull request:

    • Create a new branch for this feature
    • Commit changes with clear, descriptive commit messages
    • Push the branch to the repository
    • Create a pull request with a detailed description of the changes

17. Address review feedback:

    • Respond to any comments or questions on the pull request
    • Make necessary adjustments based on reviewer feedback
    • Re-run tests after making changes to ensure everything still works

18. Finalize the implementation:

    • Ensure all CI/CD checks pass
    • Squash commits if required by the project guidelines
    • Merge the pull request once approved

[0xdevcollins]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

My name is Collins Ikechukwu. I am a fullstack developer with 4 years of experience. I'm thrilled about the chance to contribute to the Op Scan Project. With 4 years of solid experience in Next.js, TypeScript, Shadcn, and React, Nodejs and PHP I'm confident in my ability to deliver an efficient and user-friendly Contract page for transactions.

How I plan on tackling this issue

Create fetch-blocks.ts

I will Create a new file components/pages/blocks/fetch-blocks.ts.
Define a top-level exported function that abstracts the data source. This function will decide whether to fetch data from JSON-RPC or the database based on an environment variable.

Implement fetchFromDatabase

I will create or update lib/fetch-database.ts to include the function for querying blocks from the database using Prisma. Ensure that pagination logic is preserved.

Update lib/fetch-data.ts

I will Move the fetchLatestBlocks function logic from lib/fetch-data.ts to components/pages/blocks/fetch-blocks.ts. Ensure it is used appropriately.

Update Blocks Page Component

I will In the top-level server component for the blocks page, use the new fetchBlocks function to retrieve data.

[PoulavBhowmick03]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

I'm Poulav Bhowmick, with a robust background in TypeScript, fullstack development and blockchain technology. My experience includes building robust applications, optimizing functionalities and blockchain integration. I have actively participated in events and open source contributions, enhancing my capability to tackle real-world tech challenges. My projects can be viewed on my GitHub Profile and OnlyDust Profile. Plus I´m active member of Starknet community🇷. This is my first contribution to Walnut’s Repositories.

How I plan on tackling this issue

To implement fetching latest blocks from either JSON-RPC or a database, I propose the following steps:

1. Create a new file components/pages/blocks/fetch-blocks.ts with a top-level exported function fetchLatestBlocks.

2. Implement two separate functions:

   • fetchLatestBlocksFromJsonRpc: Move the existing logic from lib/fetch-data.ts.
   • fetchLatestBlocksFromDatabase: Implement new logic using Prisma to query the database.

3. In the top-level fetchLatestBlocks function, use an environment variable (e.g., DATABASE_URL) to determine which data source to use, similar to the transaction details implementation.

4. Ensure that the pagination logic is preserved when querying data via Prisma in the database function.

5. Update the blocks page to use the new fetchLatestBlocks function instead of calling fetchLatestBlocks from lib/fetch-data.ts.

6. Implement error handling and fallback to JSON-RPC if the database query fails.

Here's a skeleton of the fetch-blocks.ts file:

import { l2PublicClient } from "@/lib/chains";
import { prisma } from "@/lib/prisma";
import { fromViemBlock, Block } from "@/lib/types";
import { range } from "@/lib/utils";

const fetchLatestBlocksFromJsonRpc = async (start: bigint): Promise<Block[]> => {
  const blocksPerPage = BigInt(process.env.NEXT_PUBLIC_BLOCKS_PER_PAGE);
  const blocks = await Promise.all(
    range(Number(start), Math.max(Number(start - blocksPerPage), -1)).map((i) =>
      l2PublicClient.getBlock({ blockNumber: BigInt(i) }),
    ),
  );
  return blocks.map(fromViemBlock);
};

const fetchLatestBlocksFromDatabase = async (start: bigint): Promise<Block[]> => {
  const blocksPerPage = BigInt(process.env.NEXT_PUBLIC_BLOCKS_PER_PAGE);
  const blocks = await prisma.block.findMany({
    where: {
      number: {
        lte: start.toString(),
        gt: (start - blocksPerPage).toString(),
      },
    },
    orderBy: {
      number: 'desc',
    },
    take: Number(blocksPerPage),
  });

  if (blocks.length === 0) {
    return fetchLatestBlocksFromJsonRpc(start);
  }

  return blocks.map(block => ({
    number: BigInt(block.number),
    hash: block.hash,
    parentHash: block.parentHash,
    timestamp: BigInt(block.timestamp),
    nonce: block.nonce,
    difficulty: BigInt(block.difficulty),
    gasLimit: BigInt(block.gasLimit),
    gasUsed: BigInt(block.gasUsed),
    miner: block.miner,
    extraData: block.extraData,
    baseFeePerGas: block.baseFeePerGas ? BigInt(block.baseFeePerGas) : null,
    transactions: [], // Assuming we don't need transactions here, adjust if needed
  }));
};

const fetchLatestBlocks = process.env.DATABASE_URL
  ? fetchLatestBlocksFromDatabase
  : fetchLatestBlocksFromJsonRpc;

export default fetchLatestBlocks;

This implementation follows the pattern established in the transaction details fetching, using the DATABASE_URL environment variable to determine the data source. It also includes a fallback to JSON-RPC if the database query fails or returns no results.
The blocks page should be updated to use this new function instead of calling fetchLatestBlocks from lib/fetch-data.ts.

[martinvibes]

I am applying to this issue via OnlyDust platform.

hello @saimeunt i would love to be assigned on this issue and be a ccontributor

My background and how it can be leveraged

i'm a **frontend developer** and i have experience using javascript, react, tailwind css, typescript etc.

How I plan on tackling this issue

Steps:
Set Up Your Environment:

Clone the op-scan repository and set up your .env file with the DATA_SOURCE variable set to either jsonrpc or database.
Create the Helper Function:

Create fetch-blocks.ts under components/pages/blocks/.
Implement functions for fetching data from JSON-RPC and the database.
Export a top-level function to abstract the data source based on the environment variable.
Move Existing Logic:

Move the block fetching logic from lib/fetch-data.ts to fetch-blocks.ts.
Update the Blocks Page:

Modify the blocks page to call the new helper function.
Run op-indexer:

Set up and run op-indexer to keep the database synchronized.
Test the Implementation:

Test fetching blocks from both JSON-RPC and the database.
Verify data consistency and pagination.

i would love to participate on this repo and be amongs other contributors

[danielcdz]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

Hello! I worked in the last ODHack by adding the latest 10 transaction component in the home page, this issue seems to be pretty similar to it but now the data source is the DB.

How I plan on tackling this issue

I will follow what you suggested, basically querying the data from the DB and showing the results in the existing component, since I already worked on this project in the past I already have enough context :)

[johnkennedyb]

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

My name is Nnawuihe Johnkennedy .I am a software developer using Mern stack, i havea wealth of experiences including working for Thorium Logic ,Ivan research institute e.t .c .This is my portfolio :johnkennedy.vercel.app

How I plan on tackling this issue

I can create a fetch-blocks.ts helper function to fetch latest blocks data from either JSON-RPC or a database using Prisma, configured via an environment variable, ensuring pagination logic is preserved, and update the blocks page to use this function while running op-indexer to keep the database updated.

[onlydustapp]

The maintainer saimeunt has assigned Ugo-X to this issue via OnlyDust Platform.
Good luck!