INTERNSHIP- Rawan Mostafa

[RawanMostafa08]

intro

This is a rough plan of the development internship program for 2024 -8 to 10 weeks program-. The intern need to be able to reason about the code, and be able to write tests (units and integration), then they should be able to contribute to the daily work with the team.

Week 0

In this week we spend some time to get familiar with the basics of linux, git, shell, and how to read the documentation

• https://www.youtube.com/playlist?list=PLyzOVJj3bHQuloKGG59rS43e29ro7I57J ( lect 1-7)
• Effective Shell: https://effective-shell.com
• Linux Intro/Essentials https://lym.readthedocs.io/en/latest/ (till Linux Services, package management (apt only), networking commands)
• Reading manpages of echo, env, cat, wc, head, tail, yes, true, false, tree
• git https://www.atlassian.com/git (branches, commit, log, merge, rebase, tag, github flow)

extra

• audit courses https://www.coursera.org/professional-certificates/google-it-support
• https://ubuntu.com/server/docs

Week 1

We get our feet with learning the fundamentals of Go, and start developing small utilities in Go.

• Go fundamentals gobyexample
• coreutils Coreutils - Template #105: echo, env, cat, wc, head, tail, yes, true, false, tree (use gobyexample to speed up things)
• build your own git https://leshenko.net/p/ugit/# (5 steps)

Week 2

We introduce you to the testing, writing unit tests, configuring github workflows.

• learn Go with tests
• INI Parser project with tests INI Parser - template  #21
• Github Workflows
• build your own git https://leshenko.net/p/ugit/# (5 steps)

Week 3

In this week we develop a small http service and we cover how to test it and how to build a client to consume its api

• Datetime server Datetime Server - Template #144
• HTTP Service testing
• DateTime http client Datetime HTTP Client - Template #145
• context package
• build your own git https://leshenko.net/p/ugit/# (5 steps)

Extra

• go101
• ultimatego

Week 4

In this week we get familiar with docker and docker compose, what they do, and how to build a docker image, and how to reduce its size. Then we go through REST API, Swagger, and Postman. and we will build a small client that consumes the pokemon Restful API service.

• Docker
• Multistaged builds https://fabianlee.org/2020/01/26/golang-using-multi-stage-builds-to-create-clean-docker-images/
• Docker compose
• REST API
• Swagger
• Postman
• pokemon API https://pokeapi.co/ (create http client for pokemon API + tests) PokeAPIHTTPClient - Template #146
• build your own git https://leshenko.net/p/ugit/# (5 steps)

Resources

• docker docs https://docs.docker.com/engine/install/ubuntu/
• https://www.youtube.com/watch?v=3c-iBn73dDE
• https://www.youtube.com/watch?v=pTFZFxd4hOI
• https://www.youtube.com/watch?v=HG6yIjZapSA
• https://www.youtube.com/watch?v=PrusdhS2lmo
• https://www.youtube.com/watch?v=X48VuDVv0do
• docker docs https://docs.docker.com/engine/install/ubuntu/
• https://www.youtube.com/watch?v=3c-iBn73dDE
• https://www.youtube.com/watch?v=pTFZFxd4hOI
• https://www.youtube.com/watch?v=HG6yIjZapSA
• https://www.youtube.com/watch?v=PrusdhS2lmo
• multistaged builds https://fabianlee.org/2020/01/26/golang-using-multi-stage-builds-to-create-clean-docker-images/
• https://www.youtube.com/watch?v=X48VuDVv0do

Week 5

Here we introduce you to the concept of MVC.

• Secret Note MVC Secret Note - MVC - Template #153
• build your own git https://leshenko.net/p/ugit/# (5 steps)

Week 6

In this week we will redo the project as SPA, by providing two services: an API service and a frontend that consumes that API -next week-

• Secret Note API Secret note (API/SPA)- Template #152

Week 7

• Frontend Secret note (API/SPA)- Template #152
• build your own git https://leshenko.net/p/ugit/# (5 steps)

Resources

• Frontend track https://github.com/MohamedElmdary/cat-frontend-circle-road-map

• https://github.com/MohamedElmdary/frontend-session-resources

• Vue.js

• React Megatutorial

• https://selenium-python.readthedocs.io/locating-elements.html

• https://www.toolsqa.com/selenium-webdriver/selenium-locators/

• https://filiphric.com/cypress-basics-selecting-elements

• Cypress https://testautomationu.applitools.com/advanced-cypress-tutorial/

• Testing session: https://docs.google.com/presentation/d/1ljUyCXWN-ZF3FyNDUOYXoyOrEagy_DoXWeL5wDMF47Y/edit?usp=sharing

Week 8 (onboarding to threefold)

Starting from that week the intern should be able to write testable code and can be introduced to our platform and projects. Best intro the platform by manually testing the capabilities and reporting issues on the normal of the enduser.

• Intro to Codescalers and TFGrid Platform
• How to write a bug report https://www.softwaretestinghelp.com/how-to-write-good-bug-report/
• Manual Testing
• Namespaces, CGroups, Docker
• The Dashboard and deploying your first machine on the threefold grid)
• Terraform and deploying your virtual machines machines on the threefold grid)
• Rust fundamentals
• Manually Testing the platform (intro to testlodge -qa tool we use to develop and track our tests-)

Week 9-end of internship week 10 distributed on teams (sdk-go, zos, sdk-ts, sdk-dart)

• 4 solutions
• GPU services integration *
• ZOS service rewrites in rust
• Freeflow twin
• Threefold Connect *
• Planetary Desktop/mobile App
• Increase code coverage
• Deployments web builder (drag/drop)
• IaC
• heroku like system on the grid maybe? gridify pushing forward

Extra tasks

• RESP Parser RESP parser #275
• dotenv dotenv - Template #208
• Makego Makego - Template #157
• Busybox img busybox image #103
• NTP Minimal NTP Client - Template #141
• BMP Width/Height Extract Height and Width from Bitmap Image - Template  #142
• Msgpack Msgpack - Template #240
• bitcask paper

Sessions

• REST API
• GraphQL
• Protobuf
• gRPC
• Concurrency
• Blockchain
• Vue
• Docker/CGroups
• Testing
• Redis
• UNIX Programming
• Caddy
• Zinit
• Flist
• Raft
• Packer
• Zig
• TLS
• Encryption (symmetric/asymmetric)
• 12 factor apps https://12factor.net
• Github flow https://docs.github.com/en/get-started/quickstart/github-flow
• Building a small busybox based distro

daily work

• write your own git: https://leshenko.net/p/ugit/#
• creating package in 2022
• python
• django rest framework DRF
• pymotw
• jinja
• bottle
• exploreflask
• flask megatuorial
• TDD with python/django
• redis university
• redis in action is a book that will help you to know redis by practice.
• The following topics are important to know about Redis:
  • Data structures
  • Pub/Sub Guide link 1 | Guide link 2 | Video
  • Data persistence (snapshots & AOF) Guide link | Video
  • Transactions Guide link
  • Distributed locking Guide link1 | Guide link2
  • Resp Protocol Guide link

Software development process / testing

• Trunk based development https://trunkbaseddevelopment.com/

networking

• CCNA/CCNP

linux

• effective shell: https://effective-shell.com
• ubuntu server guide
• nginx
• debian handbook
• tmux
• building root filesystem
• coredns
• https://www.nginx.com/blog/what-are-namespaces-cgroups-how-do-they-work/
• https://www.youtube.com/watch?v=8fi7uSYlOdc
• build container in 500Lines
• unix programming

Raft/Distributed systems

• https://learn.particular.net/courses/distributed-systems-design-fundamentals-online
• slides: https://thesecretlivesofdata.com/raft/
• https://raft.github.io
• https://en.wikipedia.org/wiki/Raft_(algorithm)

binary serialization

• https://capnproto.org/
• https://google.github.io/flatbuffers/
• https://developers.google.com/protocol-buffers

Optional:

• Kubernetes
• Helm Charts
• Blockchain (substrate, cosmos, solidity?)

[RawanMostafa08]

1 Sep 2024

• Updated my tickets by the 2nd batch updates
• Implemented Unix coreutils with the needed flags (will take another look tomorrow to add documentation and make sure everything is working before the review)
• Looked at go by example and https://cheat.sh/tree when needed
• Struggled a bit with git config to switch back to my old user 😂

[RawanMostafa08]

2 Sep 2024

• Finalized the coreutils (added error messages, uploaded binary files, added a Readme and opened a PR to merge development to main and labeled it ready for review)
• Started the build your own Git tutorial, created the venv and needed setup, reached step 4 but had a blocker which we agreed to fix tomorrow Isa
• Started reading about redis publisher and subscriber, finished Guide link 1 and this Video

[RawanMostafa08]

3 Sep 2024

• Had a code review round and a session in the office
• Fixed the issue in step 4 of build you own git
• Applied some of the review comments

[RawanMostafa08]

4 Sep 2024

• Applied coreutils review comments
• Added some extra utils like true false yes env
• Added github actions workflow for (linting, formatting, release)
• Completed the last step of build your own git (5 steps of the week are done)
• Read "Software Design by Example" Chapter 1 https://third-bit.com/sdxpy/intro/

[RawanMostafa08]

5 Sep 2024

• Finished learn go with tests till Arrays and slices
• Made the workflow of the release triggered on tag creation using Goreleaser

[RawanMostafa08]

7 Sep 2024 [weekend]

• Watched Go Testing Bible videos till [10 - Getting Started with Testify]
• Finished learn go with tests till Mocking [dropped pointer & errs / maps sections]

[RawanMostafa08]

8 Sep 2024

• Finished learn go with tests pointer & errs
• Read INI files documentation and took a look on config parser docs
• Applied coreutils 2nd round review comments
• Made first version of LoadFromString function and its test for INI parser

[RawanMostafa08]

9 Sep 2024

• Finished INI parser LoadFromString LoadFromFile GetSectionNames GetSections Get(section_name, key) and their tests
• Added workflow files for github actions [not passing will try it again tomorrow]
• Applied 3rd round review comments for coreutils

[RawanMostafa08]

10 Sep 2024

• Finished INI parser functions, tests, API documentation and user documentation
• Successfully added workflows for github actions

[RawanMostafa08]

11 Sep 2024

• Added testcases and refactoring for INI parser
• Finished building your own git till step 10 ✅
• Read learn go with tests [ maps, dependency injection, mocking] to apply some of the concepts to the ini code

[RawanMostafa08]

17 Sep 2024

• Implemented the stringer interface for the ini parser
• Implemented a simple http datetime server
• Added testcases for the GetDate handler function in the http server
• Implemented the server using Gin and added its testcases
• Implemented a simple http client

[RawanMostafa08]

18 Sep 2024

• Watched till lesson 6 of docker from kodecloud
• Added plain text and json handling for http, gin servers
• Added github workflows for server
• Made the url,port,endpoint configurable using flags for client

[RawanMostafa08]

19 Sep 2024 [progress till now] [Will continue in 2 hours Isa]

• Added test cases for http, gin servers
• Made the unit tests
• Edited the code dependencies and organization
• Added env var configuration in client
• Added github workflows for client
• Added docker file for server (not finished)

[ToDos later today&weekend Isa]

• Build your own get (5 steps) [commit: Record hash of last commit to HEAD]
• Continue the docker files for server
• Add docker compose
• Add retry mechanism for client
• Make documentation for client and server
• Logging
• Makefile
• mock server test for client

[RawanMostafa08]

22 Sep 2024

• Applied most of INI parser review comments
• Made a first version of pokemon API client which will need huge revisit because found out I'm making the client concept wrong
• Fixed the HTTP client to match the new understanding of the client ( made the client handle all http logic, return time, edited its tests to match the change)
• pushed the docker images of the two servers to docker hub and pulled them in the client side for github actions to pass in case of testing the local server (not the mock)

[RawanMostafa08]

23 Sep 2024

• Finished the PokeAPI client (Resources and pokemon APIs, their tests, workflow file, documantation, retry mechanism, configuration, logging)
• Read "Software Design by Example" Chapter 2 https://third-bit.com/sdxpy/oop/ [Will continue reading later tonight Isa]

Update

• Read "Software Design by Example" Chapter 3
• Read "Software Design by Example" Chapter 4
• Read "Software Design by Example" Chapter 5

[RawanMostafa08]

24 Sep 2024

• Finished applying the review comments of INI parser
• finished 2 steps of build your own git
• Attended the dependency injection, uber go and kubernetes sessions at the office

[RawanMostafa08]

25 Sep 2024

• Watched a Swagger tutorial
• Implemented the CPU and Mem utils for psutils project and their tests
• Read the Mocking chapter of learn go with tests again
• Implemented Dependency injection in the psutils test
• started working on process util but not finished

[RawanMostafa08]

26 Sep 2024

• Implemented the get process list and get process details APIs in psutils and their tests
• Prepared some of the session material, agenda and slides

[RawanMostafa08]

29 Sep 2024

• Applied datetime server review comments
• Read about kubernetes from the documantation

[RawanMostafa08]

30 Sep 2024

• Watched this kubernetes tutorial https://www.youtube.com/watch?v=s_o8dwzRlu4&t=104s
• Applied the rest of the http server review comments
• Created two deployments for http and gin servers on the cluster given to us
• Created two services for http and gin
• Remaining: figure out how to have an external ip to test the app

[RawanMostafa08]

1 Oct 2024

• Finished deploying the app on Kubernetes
• Applied http client review comments
• Watched this tutorial for helm charts : https://www.youtube.com/watch?v=-ykwb1d0DXU

[RawanMostafa08]

2 Oct 2024

• Watched this tutorial for helm charts: https://www.youtube.com/watch?v=jUYNS90nq8U
• Created the helm chart for the datetime server
• Made Helm templating for values in the yaml files
• finished 3 steps of build your own git

[RawanMostafa08]

3 Oct 2024

• finished 5 steps of build your own git

[RawanMostafa08]

7 Oct 2024

• Watched this MVC playlist: https://www.youtube.com/watch?v=pCvZtjoRq1I&list=PLtzRmxotRNsNRx0UgjIXuR38xEGSTUoT6&index=1
• Watched this tutorial for django: https://www.youtube.com/watch?v=rHux0gMZ3Eg until templates and applied with it

[RawanMostafa08]

8 Oct 2024

• Finished this tutorial for django: https://www.youtube.com/watch?v=rHux0gMZ3Eg

[RawanMostafa08]

9 Oct

• Followed this django docs tutorial https://docs.djangoproject.com/en/5.1/intro/tutorial01/ to make the app and made the Note model, view note view

[RawanMostafa08]

10-11-12 Oct 2024

• Session preparations (encryption topic), slides preparation, research, videos, demo code making
• Worked a bit on django project functionality (Secret note MVC)

[RawanMostafa08]

13 Oct 2024

• Attended the sessions and gave my session in the office

[RawanMostafa08]

14 Oct 2024

• Implemented the authentication of secret note app
  - Signup
  - Login
• Implemented view user notes
• Implemented create note form and view

[RawanMostafa08]

15 Oct 2024

• implemented the note deletion logic on number of views and expiry date

[RawanMostafa08]

16 Oct 2024

• Added styles for templates
• Added unit tests for login and signup

[RawanMostafa08]

17 Oct 2024

• Added the rest of the unit tests
• Added the rate limiting and its unit tests

[RawanMostafa08]

18 Oct 2024 [weekend]

• watched this tutorial for django testing with selenium https://www.youtube.com/watch?v=XnSQ6sRGKzI

[RawanMostafa08]

19 Oct [weekend]

• watched python selenium tutorials
• Added end to end tests with selenium (was having some issues)
• Containerized the app using docker, docker-compose
• Added github workflow file (was having some issues)

[RawanMostafa08]

20 Oct

• Fixed the issues with github workflows
• Fixed the issues with the integration tests
• Added README file for secret note app

[RawanMostafa08]

21 Oct

• Implemented a part of the authentication logic of the linktree application following this https://www.youtube.com/playlist?list=PL5dTjWUk_cPY7Q2VTnMbbl8n-H4YDI5wF

[RawanMostafa08]

22 Oct - 23 Oct

• Continued the JWT authentication logic including hashing the passwords using bcrypt before storing in db and validation when retrieving
• Added unit tests for login and signup handlers
• Adding docker-compose to use mongo image
• Adding make file

[RawanMostafa08]

24 Oct [Thursday and the weekend]

• Added Add link API
• Added edit link API
• Added delete link API
• Added unit tests for the mentioned APIs

[RawanMostafa08]

Sun 27 Oct

• Added get user links API
• Its unit tests

[RawanMostafa08]

Mon 28 Oct

• Added swagger API documentation
• Started watching this tutorial for vue.js https://www.youtube.com/watch?v=1GNsWa_EZdw