Add log streaming to the created job

[amritghimire]

When running a job from studio, this adds a log streaming which checks
for the updates from the Studio.

Related PR: https://github.com/iterative/studio/pull/11068

[cloudflare-workers-and-pages]

Deploying datachain-documentation with    Cloudflare Pages

Latest commit:	a19394b
Status:	✅  Deploy successful!
Preview URL:	https://b8e72553.datachain-documentation.pages.dev
Branch Preview URL:	https://amrit-stream-logs.datachain-documentation.pages.dev

View logs

[codecov]

Codecov Report

Attention: Patch coverage is 72.09302% with 12 lines in your changes missing coverage. Please review.

Project coverage is 87.38%. Comparing base (10e90c5) to head (a19394b).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/datachain/studio.py	54.54%	7 Missing and 3 partials ⚠️
src/datachain/remote/studio.py	90.47%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #680      +/-   ##
==========================================
- Coverage   87.44%   87.38%   -0.07%     
==========================================
  Files         114      114              
  Lines       10898    10939      +41     
  Branches     1499     1506       +7     
==========================================
+ Hits         9530     9559      +29     
- Misses        990      999       +9     
- Partials      378      381       +3     

Flag	Coverage Δ
datachain	87.32% <72.09%> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dreadatour]

Looks good to me! Thank you for this update 🥰

[0x2b3bfa0]

Still not sure WebSockets are a good idea, especially for log streaming. I'm not even talking about SSE here, but even about plain old polling.

[0x2b3bfa0]

It works, but I wonder if it should be closer to real-time; the following example seems to be buffering the whole output and takes ages to print the logs: 🧐

datachain studio run test.py

import time

for number in range(60 * 60):
  time.sleep(0.1)
  print(number)

[amritghimire]

It works, but I wonder if it should be closer to real-time; the following example seems to be buffering the whole output and takes ages to print the logs: 🧐

datachain studio run test.py

import time


for number in range(60 * 60):
  time.sleep(0.1)
  print(number)

Found an possible cause in Studio side. Working on it alongside the test cases for this.