Refresh data inside table with timely intervals

[vigne-Sh]

Follow up of discussions/3328

• I wanted to check whether we can

  • Allow the row values inside Table to refresh periodically by fetching the data through API service calls

• From the suggestions on above mentioned discussion, I tried mocking the service data with set_interval

import requests
import random

from rich.table import Table
from rich.console import Console
from textual.app import App

from textual.containers import  ScrollableContainer
from textual.widgets import (
    Header,
    Static,
)

def get_table_data() -> dict:
    """
    actual table which fetches data from the service call
    """
    service_url = "<service_url>"
    response = requests.get(service_url)
    return response.json()

def get_mocked_table() -> Table:
    """
    mocked table similar to actual service call response
    """
    col1 = [str(random.random()) for i in range(30)]
    col2 = [''.join(random.choices(['completed', 'pending'])) for i in range(30)]
    table = Table(title='test dynamic table')
    table.add_column('col 1 numerical')
    table.add_column('col 2 string values')
    for i, j in zip(col1, col2):
        table.add_row(i, j)
    console = Console()
    console.print(table)
    return table

class DynamicData(App):

    def compose(self):
        yield Header(show_clock=True, id="app_header")
        yield ScrollableContainer(Static(get_mocked_table(), id='dynamic_container'))

    def on_mount(self) -> None:
        self.set_interval(2, self.update_data)

    def update_data(self):
        container = self.query_one('#dynamic_container', Static)
        container.update(get_mocked_table())

DynamicData().run()

• I was able to achieve my requirement, but is this the recommended way to do .? , specially if the no of tables to render are more than 7 - 10

• Will there be any impact in the performance when doing live rendering ?

[willmcgugan]

You won't need to construct Console objects (or print) in Textual.

Be careful when using the requests module in Textual. requests doesn't support async, which means it will pause the UI. Better to use httpx, which support async out of the box.

Otherwise, there is nothing wrong with your approach. Putting a Rich table in a widget is totally fine, but we do have a dedicated DataTable widget that integrates better with Textual.

[TomJGooding]

Sorry Will but just to confirm, there's nothing wrong with using a worker with set interval? #3328 (reply in thread)

[willmcgugan]

SHould be fine. Timer callbacks can be async.

[vigne-Sh]

I tried using httpcore instead of requests to make the http client calls, I had 2 more queries to ask.

I did read your blog about async

1 - Is normal HTTP calls not compatible with textual UI when compared to async

• In the Below app, I tired using the same http requests in normal & async mode using the same httpcore

• The UI was freezing each time when table is updated using the normal function call, to see the UI Freeze I added the timer on the top right which gets into frozen state during each refresh

from textual.containers import  ScrollableContainer
from textual.widgets import (
    Header,
    Static,
)
import pandas as pd
import json
import httpcore
import time
import numpy as np

service_url = "https://jsonplaceholder.typicode.com/todos"

def get_table_data() -> Table:
    """
    actual table which fetches data from the service call
    """
    response = httpcore.request('GET', service_url)
    time.sleep(3)
    df = pd.DataFrame(json.loads(response.content))
    # shuffling the values
    df = df.iloc[np.random.permutation(len(df))]
    table = Table(title='sync-test dynamic table')
    table.add_column('userId')
    table.add_column('Title')
    table.add_column('Completed')
    for i, j, k in zip(df.userId, df.title, df.completed):
        table.add_row(f'{i}', f'{j}', f'{k}')
    return table

async def get_async_table_data() -> Table:
    """
    fetches data from the service call in async mode
    """
    async with httpcore.AsyncConnectionPool() as http:
        response = await http.request("GET", service_url)
        await asyncio.sleep(3)
        df = pd.DataFrame(json.loads(response.content))
        # shuffling the values
        df = df.iloc[np.random.permutation(len(df))]
        table = Table(title='async-test dynamic table')
        table.add_column('userId')
        table.add_column('Title')
        table.add_column('Completed')
        for i, j, k in zip(df.userId, df.title, df.completed):
            table.add_row(f'{i}', f'{j}', f'{k}')
        return table

def empty_table():
    table = Table(title='empty table')
    table.add_column('userId')
    table.add_column('Title')
    table.add_column('Completed')
    return table

class DynamicData(App):

    def compose(self):
        yield Header(show_clock=True, id="app_header")
        yield ScrollableContainer(Static(empty_table(), id='dynamic_container2'))
        yield ScrollableContainer(Static(empty_table(), id='dynamic_container3'))

    def on_mount(self) -> None:
        self.set_interval(5, self.notify_refresh)
        self.set_interval(9, self.update_data)

    def notify_refresh(self):
        self.notify(
                "Refreshing Table Data in few seconds...",
                title="Refresh Alert",
                severity="information",
                timeout=2,
            )

    async def update_data(self):
        container2 = self.query_one('#dynamic_container2', Static)
        container3 = self.query_one('#dynamic_container3', Static)
        container2.update(get_table_data())
        container3.update(await get_async_table_data())

DynamicData().run()

• Whereas if I comment-out and use only the get_async_table_data function, there is no UI freeze observed and the table values are refreshed smoothly

from textual.containers import  ScrollableContainer
from textual.widgets import (
    Header,
    Static,
)
import pandas as pd
import json
import httpcore
import time
import numpy as np

service_url = "https://jsonplaceholder.typicode.com/todos"

async def get_async_table_data() -> Table:
    """
    actual table which fetches data from the service call
    """
    async with httpcore.AsyncConnectionPool() as http:
        response = await http.request("GET", "https://jsonplaceholder.typicode.com/todos")
        await asyncio.sleep(3)
        print(response.content)
        df = pd.DataFrame(json.loads(response.content))
        # shuffling the values
        df = df.iloc[np.random.permutation(len(df))]
        table = Table(title='async-test dynamic table')
        table.add_column('userId')
        table.add_column('Title')
        table.add_column('Completed')
        for i, j, k in zip(df.userId, df.title, df.completed):
            table.add_row(f'{i}', f'{j}', f'{k}')
        console = Console()
        console.print(table)
        return table

def empty_table():
    table = Table(title='empty table')
    table.add_column('userId')
    table.add_column('Completed')
    return table

class DynamicData(App):

    def compose(self):
        yield Header(show_clock=True, id="app_header")
        yield ScrollableContainer(Static(empty_table(), id='dynamic_container3'))

    def on_mount(self) -> None:
        self.set_interval(5, self.notify_refresh)
        self.set_interval(9, self.update_data)

    def notify_refresh(self):
        self.notify(
                "Refreshing Async TableData in few seconds...",
                title="Refresh Alert",
                severity="information",
                timeout=2,
            )

    async def update_data(self):
        container3 = self.query_one('#dynamic_container3', Static)
        container3.update(await get_async_table_data())

DynamicData().run()

• So is it mandatory to use async for all the API calls that are made for refreshing the table without freezing the UI ?

---

2 - When I Try to place the get_async_table_data() which returns a rich Table inside the ScrollableContainer, am getting

Type "Coroutine[Any, Any, Table]" cannot be assigned to type "RenderableType"
    "Coroutine[Any, Any, Table]" is incompatible with protocol

 

• I had 2 options

  • one is to render a empty table when the app is starting & then update the latest data on refresh

  yield ScrollableContainer(Static(empty_table(), id='dynamic_container2'))
  

  • Another is to get the get_async_table_data return to a variable using asyncio.run inside the App class, but this causes delay to load the App with blank screen

  table_data = asyncio.run(get_async_table_data())
  
  def compose(self):
         yield ScrollableContainer(Static(table_data, id='dynamic_container2'))
  

  • Correct. me if am wrong, Is there any other options to place the Table Data that is generated using async calls ?

  ---

[davep]

@vigne-Sh I suspect you may find the worker API helpful here.

[vigne-Sh]

Thanks alot 💯, I did see the Worker docs while doing the textual tutorial some time back, but forgot to use it totally.

Adding @work(exclusive=True, thread=True) helped the synchronous function calls to work smoothly without any issues