Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert blocks/ and generators/ to TypeScript #6828

Closed
69 of 72 tasks
Tracked by #7652
cpcallen opened this issue Feb 7, 2023 · 8 comments
Closed
69 of 72 tasks
Tracked by #7652

Convert blocks/ and generators/ to TypeScript #6828

cpcallen opened this issue Feb 7, 2023 · 8 comments
Assignees
Labels
component: generators component: library blocks issue: bug Describes why the code or behaviour is wrong

Comments

@cpcallen
Copy link
Contributor

cpcallen commented Feb 7, 2023

This is a tracking bug for the remaining work to migrate the Blockly codebase to TypeScript, following the successful migration of core/.

This is part of #5857, and will resolve #6248 and probably also #2995.

Work to be done

  • Update scripts/gulpfiles/build_tasks.js to feed the contents of blocks/ and generators/ through tsc.
  • Update tests/bootstrap.js, playgrounds, etc. to load blocks and generators from build/src/ when loading uncompressed.
    • Turned out to be ~no work for now as they use goog.require so don't care about file location.
  • Create script to assist with migration (esp. of imports).
  • [OPTIONAL:] Refine type of BlockDefinition (in core/blocks.ts) to make defining blocks easier and preferably less error prone.
    • At the moment BlockDefinition is AnyDuringMigration; this works but means that tsc provides no assistance in verifying block definitions—for example, it does not check to make sure that Block methods are being called (or overridden) correctly.
    • @cpcallen has asked a question on Stack Overflow but this has not elicited an answer yet.
    • @btw17 has made some useful suggestions, including in particular defining BlockDefinition as {[Property in keyof Block]?: Block[Property]} & {[key: string]: unknown}, but this is not perfect: e.g. mixin methods do not get the correct type for this.
  • Create a model PR, migrating one file in blocks/, to verify that the above works as intended.

blocks/

generators/

  • Provide more user-friendly generator APIs #7326

  • Make CodeGenerator abstract #7401

  • dart.js

    • dart/dart_generator.js
    • dart/colour.js
    • dart/lists.js
    • dart/logic.js
    • dart/loops.js
    • dart/math.js
    • dart/procedures.js
    • dart/text.js
    • dart/variables.js
    • dart/variables_dynamic.js
  • javascript.js

    • javascript/javascript_generator.js
    • javascript/colour.js
    • javascript/lists.js
    • javascript/logic.js
    • javascript/loops.js
    • javascript/math.js
    • javascript/procedures.js
    • javascript/text.js
    • javascript/variables.js
    • javascript/variables_dynamic.js
  • lua.js

    • lua/lua_generator.js
    • lua/colour.js
    • lua/lists.js
    • lua/logic.js
    • lua/loops.js
    • lua/math.js
    • lua/procedures.js
    • lua/text.js
    • lua/variables.js
    • lua/variables_dynamic.js
  • php.js

    • php/php_generator.js
    • php/colour.js
    • php/lists.js
    • php/logic.js
    • php/loops.js
    • php/math.js
    • php/procedures.js
    • php/text.js
    • php/variables.js
    • php/variables_dynamic.js
  • python.js

    • python/python_generator.js
    • python/colour.js
    • python/lists.js
    • python/logic.js
    • python/loops.js
    • python/math.js
    • python/procedures.js
    • python/text.js
    • python/variables.js
    • python/variables_dynamic.js
@rachel-fenichel
Copy link
Collaborator

I took a quick crack at converting some of the generators files to see what broke. What I found:

  • Individual language + block files (e.g. generators/javascript/colour.js) require the language generator file (generators.javascript.js) rather than the other way round.
    • That may mean we have to convert the language generator files first.
  • Each language generator is declared as a new CodeGenerator('name') and then adds properties in several ways:
    • By calling specific functions (e.g. addReservedWords)
    • By overriding protected functions (e.g. quote_)
    • By adding functions (e.g. JavaScript.finish = function(code) { ... })
    • By adding properties (e.g. JavaScript.ORDER_ATOMIC = 0)
  • The language generators also access properties that are protected, such as nameDB_.
    • This causes a warning, because as protected properties those should only be touched by subclasses, and the generator is an instance rather than a subclass.

Adding properties could be done with mixins, I think, but I'm not sure if that helps. As a test I tried something similar to what we do for blocks:

type JavaScriptGenerator = CodeGenerator&MyInterface;
interface MyInterface {
  ORDER_ATOMIC: number,
}

/**
 * JavaScript code generator.
 */
const JavaScript : JavaScriptGenerator = new CodeGenerator('JavaScript');

The declaration of JavaScript breaks with this error:

Type 'CodeGenerator' is not assignable to type 'JavaScriptGenerator'.
Property 'ORDER_ATOMIC' is missing in type 'CodeGenerator' but required in type 'MyInterface'.ts(2322)

Later, the place where I set ORDER_ATOMIC works fine (no warnings or errors).

@IlCallo
Copy link

IlCallo commented May 25, 2023

Hey there, I migrated a big chunk of procedures.js to TS for personal purpose
I'm testing it out in these days
Would you like me to send a PR for it, since it seems unassigned?

I used abstract and anonymous classes to model the different objects and the internal state

It will need review since I miss most of the context
There are many usages of values which TS mention being string | null, which are used as if they are always defined
In other places there are checks for vars which should always be defined
And there are some parts of the code which don't really make sense after you add the types which are supposed to describe those snippets

@cpcallen
Copy link
Contributor Author

Hi @IlCallo! As it happens I actually have a 90% complete PR for blocks/procedures.js in progress, but I'd certainly be interested in seeing the approach you've taken because there are some things I'm not entirely happy with (in the procedures PR as well as in the ones which have been merged) and am planning to do a second pass to clean things up. (In particular, in the first pass I've used copious !s and as casts, with the goal of trying to keep the code emitted by tsc is as close as possible to the original JS, but I'd like to go back and replace most of these with runtime type assertions—at least where doing so will actually increase the robustness of the code.)

Do post a draft pull request if you like; even if not complete it would be interesting to look at.

@IlCallo
Copy link

IlCallo commented May 29, 2023

I ended up adding a lot of !s and as casts too 😁
I actually hoped you could provide some guidance about those, since I just started checking out this project and some things aren't clear to me

If you already completed most of it, I'll wait for you and add my comments/suggestions when possible, since my version is slightly customized compared to the original on in the codebase

@IlCallo
Copy link

IlCallo commented Jun 1, 2023

Would it be possible for Blockly to accept classes instead of plain object literals?
'Cause what you're doing in procedures definitions can be described by TS with classes, but object literals have structural limitations to their this type, which make difficult if not impossible to get something properly type-checked

You said you have a pending PR / branch for procedures migration, can you link it to me?

@IlCallo
Copy link

IlCallo commented Jun 1, 2023

I actually managed to get a typesafe implementation which also works (or seems to work) by using an IIFE and many this: BlockSvg annotations

Classes support would still be better IMO, but here's an example

Blocks["my_procedure_def"] = (() => {
  /* procedure internal state, accessed without this since is within scope */
  let statementConnection_: Connection | undefined | null;
  let hasStatements_: boolean | undefined;
  let arguments_: string[] = [];
  let paramIds_: string[] | undefined;
  let argumentVarModels_: VariableModel[] = [];

  /**
   * Add or remove the statement block from this function definition.
   * @param hasStatements True if a statement block is needed.
   */
  function setStatements_(this: BlockSvg, hasStatements: boolean) { // <==== NOTICE WE ARE MANUALLY TYPING THIS
    if (hasStatements_ === hasStatements) { // <==== INTERNAL STATE IS ACCESSIBLE WITHOUT "this.", THIS WORKS WITH OTHER METHODS TOO
      return;
    }
    if (hasStatements) {
      this.appendStatementInput("STACK").appendField(
        Msg["PROCEDURES_DEFNORETURN_DO"]
      );
      if (this.getInput("RETURN")) {
        this.moveInputBefore("STACK", "RETURN");
      }
    } else {
      this.removeInput("STACK", true); // <==== "this" REFERENCES ARE RELATED TO "BlockSvg" CONTEXT
    }
    hasStatements_ = hasStatements;
  }

  // ... OTHER METHODS
  
  /**
   * Block for calling a procedure with no return value.
   */
  function init(this: BlockSvg) {
    // ...
  }

  return {
    setStatements_,
    // ... other methods which need to be publicly accessible
    init,
  }
})();

Unluckily you'd then need to call all internals methods using setStatements_.call(this) to keep TS happy about the context chain being respected

@IlCallo
Copy link

IlCallo commented Jun 6, 2023

Small update: apparently wrapping the scope using functions and effectively "hiding" them from outside created some issue for which I wasn't able to pinpoint the root cause

Removing those declaration statements and updating the this declaration to account for an internal state stored into this itself seems to fix those problems

interface DefInternalState {
  arguments_: string[];
  argumentVarModels_: VariableModel[];
  statementConnection_: Connection | undefined | null;
  hasStatements_: boolean | undefined;
  paramIds_: string[] | undefined;
}

type DefBlockSvgWithInternalState = BlockSvg & DefInternalState;

Blocks["my_procedure_def"] = (() => {
  /**
   * Add or remove the statement block from this function definition.
   * @param hasStatements True if a statement block is needed.
   */
  function setStatements_(this: DefBlockSvgWithInternalState, hasStatements: boolean) { // <==== NOTICE WE ARE MANUALLY TYPING "THIS"
    if (this.hasStatements_ === hasStatements) {
      return;
    }
    if (hasStatements) {
      this.appendStatementInput("STACK").appendField(
        Msg["PROCEDURES_DEFNORETURN_DO"]
      );
      if (this.getInput("RETURN")) {
        this.moveInputBefore("STACK", "RETURN");
      }
    } else {
      this.removeInput("STACK", true); // <==== "this" REFERENCES ARE RELATED TO "BlockSvg" CONTEXT
    }
    this.hasStatements_ = hasStatements;
  }

  // ... OTHER METHODS
  
  /**
   * Block for calling a procedure with no return value.
   */
  function init(this: DefBlockSvgWithInternalState) {
    // ...
    this.arguments_ = [];
    this.argumentVarModels_ = [];
    setStatements_.call(this, true);
    this.statementConnection_ = null;
  }

  return {
    setStatements_,
    // ... other methods which need to be publicly accessible
    init,
  }
})();

@cpcallen
Copy link
Contributor Author

Closing this: the remaining OPTIONAL item has been added to #6920, and the other two items are… already items of their own. There is room to improve things but migration proper is complete even if the resulting code can certainly be improved!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: generators component: library blocks issue: bug Describes why the code or behaviour is wrong
Projects
None yet
Development

No branches or pull requests

3 participants