dart:mirrors changes broke package rpc

[jcollins-g]

Something seems to have changed after 2.1.0-dev.7.1 (Linux) with respect to type promotion in the VM, and it breaks tests in the http://github.com/dart-lang/rpc package in such a way that it is not immediately obvious what to do about it. 2.1.0-dev.7.1 passes the tests fine, but now they error out. This is now consistently breaking tests in the dart-services package as well and although I think dart-pad does not depend on this particular code path, I'm guessing other clients of the rpc package will.

To reproduce:

~/dart/clean
$ git clone git@github.com:dart-lang/rpc.git
Cloning into 'rpc'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 2224 (delta 0), reused 0 (delta 0), pack-reused 2223
Receiving objects: 100% (2224/2224), 578.11 KiB | 1.58 MiB/s, done.
Resolving deltas: 100% (1342/1342), done.
~/dart/clean
$ cd rpc
~/dart/clean/rpc
$ pub get
Resolving dependencies... (5.9s)
+ _discoveryapis_commons 0.1.8+1
+ analyzer 0.33.1
+ args 1.5.0
+ async 2.0.8
+ boolean_selector 1.0.4
+ charcode 1.1.2
+ collection 1.14.11
+ convert 2.0.2
+ crypto 2.0.6
+ csslib 0.14.6
+ dart_style 1.2.0
+ discoveryapis_generator 0.9.10
+ front_end 0.1.6+1
+ gcloud 0.6.0+1
+ glob 1.1.7
+ googleapis 0.52.0+1
+ googleapis_beta 0.47.0+1
+ html 0.13.3+3
+ http 0.11.3+17 (0.12.0 available)
+ http_multi_server 2.0.5
+ http_parser 3.1.3
+ io 0.3.3
+ js 0.6.1+1
+ json_rpc_2 2.0.9
+ kernel 0.3.6+1
+ logging 0.11.3+2
+ matcher 0.12.3+1 (0.12.4 available)
+ meta 1.1.6
+ mime 0.9.6+2
+ multi_server_socket 1.0.2
+ node_preamble 1.4.4
+ package_config 1.0.5
+ package_resolver 1.0.6
+ path 1.6.2
+ plugin 0.2.0+3
+ pool 1.3.6
+ pub_semver 1.4.2
+ quiver 2.0.1
+ shelf 0.7.3+3
+ shelf_packages_handler 1.0.4
+ shelf_static 0.2.8
+ shelf_web_socket 0.2.2+4
+ source_map_stack_trace 1.1.5
+ source_maps 0.10.8
+ source_span 1.4.1
+ stack_trace 1.9.3
+ stream_channel 1.6.8
+ string_scanner 1.0.4
+ term_glyph 1.0.1
+ test 1.4.0
+ test_api 0.1.1
+ typed_data 1.1.6
+ uri 0.11.3+1
+ utf 0.9.0+5
+ vm_service_client 0.2.6
+ watcher 0.9.7+10
+ web_socket_channel 1.0.9
+ yaml 2.1.15
Changed 58 dependencies!
Precompiling executables... (8.8s)
Precompiled discoveryapis_generator:generate.
Precompiled test:test.
~/dart/clean/rpc
$ pub run test
00:09 +59 -1: test/src/invocation/invoke_test.dart: api-invoke-post reverse-list [E]                                                                             
  Expected: <200>
    Actual: <500>
  
  package:test_api                            expect
  test/src/invocation/invoke_test.dart 635:7  main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                  _AsyncAwaitCompleter.completeError
  test/src/invocation/invoke_test.dart        main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                  _asyncThenWrapperHelper
  test/src/invocation/invoke_test.dart        main.<fn>.<fn>
  
00:09 +59 -2: test/src/invocation/invoke_test.dart: api-invoke-post add-resource [E]                                                                             
  Expected: <201>
    Actual: <500>
  
  package:test_api                            expect
  test/src/invocation/invoke_test.dart 643:7  main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                  _AsyncAwaitCompleter.completeError
  test/src/invocation/invoke_test.dart        main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                  _asyncThenWrapperHelper
  test/src/invocation/invoke_test.dart        main.<fn>.<fn>
  
00:09 +92 -3: test/api_config_test.dart: api_config_schema request-parsing [E]                                                                                   
  RPC Error with status: 400 and message: Field messages has wrong type:  type 'List<dynamic>' is not a subtype of type 'List<String>' of 'value'
  lib/src/config/schema.dart 78:9   ApiConfigSchema.fromRequest
  test/api_config_test.dart 664:21  main.<fn>.<fn>
  
00:09 +92 -4: test/api_config_test.dart: api_config_schema request-parsing-map-list [E]                                                                          
  RPC Error with status: 400 and message: Field myStrings has wrong type:  type 'List<dynamic>' is not a subtype of type 'List<String>' of 'value'
  lib/src/config/schema.dart 78:9   ApiConfigSchema.fromRequest
  test/api_config_test.dart 716:35  main.<fn>.<fn>
  
02:20 +150 -4: Some tests failed.                                                                                                                                
~/dart/clean/rpc
$

The problem seems to be that types are not being promoted the same way as they were in 2.1.0-dev.7.1, and without that I'm not sure the rpc package can be fixed. It seems to depend greatly on the prior behavior to constrain types. Manually converting the types seems to help things get farther, like in this patch (only for illustrative purposes, I don't think things like this are a good idea):

~/dart/rpc
$ git diff
diff --git a/lib/src/config/property.dart b/lib/src/config/property.dart
index a9886a3..4ba1f71 100644
--- a/lib/src/config/property.dart
+++ b/lib/src/config/property.dart
@@ -355,7 +355,15 @@ class ListProperty extends ApiConfigSchemaProperty<List, List> {
   }
 
   List _fromRequest(List encodedList) {
-    return encodedList.map(_itemsProperty._fromRequest).toList();
+    List result = encodedList.map(_itemsProperty._fromRequest).toList();
+    if (result.every((dynamic value) => value is Map)) {
+      return new List<Map>.from(result);
+    } else if (result.every((dynamic value) => value is List)) {
+      return new List<List>.from(result);
+    } else if (result.every((dynamic value) => value is String)) {
+      return new List<String>.from(result);
+    }
+    return result;
   }
 }
 
@@ -387,6 +395,13 @@ class MapProperty extends ApiConfigSchemaProperty<Map<String, dynamic>,
     encodedMap.forEach((String key, encodedObject) {
       result[key] = _additionalProperty._fromRequest(encodedObject);
     });
+    if (result.values.every((dynamic v) => v is Map)) {
+      return new Map<String, Map>.from(result);
+    } else if (result.values.every((dynamic v) => v is List)) {
+      return new Map<String, List>.from(result);
+    } else if (result.values.every((dynamic v) => v is String)) {
+      return new Map<String, String>.from(result);
+    }
     return result;
   }
 }

However even if somehow done better, the RPC package seems to depend greatly on the prior automatic constraining of types and so I don't think specifying them by hand in this way is likely to produce good results longer term.

Attempting to set breakpoints and/or use the debugger in this area crashes the VM with stack traces similar to this one:

Testing started at 1:53 PM ...
/usr/local/google/home/jcollins/dart/all_sdks/current/bin/dart --pause_isolates_on_start --enable-vm-service:35641 /usr/local/google/home/jcollins/dart/all_sdks/current/bin/snapshots/pub.dart.snapshot run test -r json /usr/local/google/home/jcollins/dart/rpc/test/api_config_test.dart
Observatory listening on http://127.0.0.1:35641/
../../runtime/platform/utils_linux.cc: 32: error: Fatal error in Utils::VSNPrint with format 'data:application/dart;charset=utf-8,%20%20%20%20import%20%22dart:isolate%22;%0A%0A%20%20%20%20import%20%22package:stream_channel/stream_channel.dart%22;%0A%0A%20%20%20%20import%20%22package:test/src/runner/plugin/remote_platform_helpers.dart%22;%0A%20%20%20%20import%20%22package:test/src/runner/vm/catch_isolate_errors.dart%22;%0A%0A%20%20%20%20import%20%22file:///usr/local/google/home/jcollins/dart/rpc/test/api_config_test.dart%22%20as%20test;%0A%0A%20%20%20%20void%20main(_,%20SendPort%20message)%20%7B%0A%20%20%20%20%20%20var%20channel%20=%20serializeSuite(()%20%7B%0A%20%20%20%20%20%20%20%20catchIsolateErrors();%0A%20%20%20%20%20%20%20%20return%20test.main;%0A%20%20%20%20%20%20%7D);%0A%20%20%20%20%20%20new%20IsolateChannel.connectSend(message).pipe(channel);%0A%20%20%20%20%7D%0A%20%20:1: Error: Can't access platform private library.'
Dumping native stack trace for thread 53a6
  [0x000055d0755e66df] dart::Profiler::DumpStackTrace(void*)
  [0x000055d0755e66df] dart::Profiler::DumpStackTrace(void*)
  [0x000055d07581e1d2] dart::Assert::Fail(char const*, ...)
  [0x000055d07581f613] Unknown symbol
  [0x000055d075520292] dart::JSONStream::PrintError(long, char const*, ...)
  [0x000055d075653fa0] Unknown symbol
  [0x000055d075650478] dart::Service::InvokeMethod(dart::Isolate*, dart::Array const&, bool)
  [0x000055d075650c05] dart::Service::HandleIsolateMessage(dart::Isolate*, dart::Array const&)
  [0x000055d0755065a3] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000055d0755381de] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000055d075538437] dart::MessageHandler::HandleOOBMessages()
  [0x000055d0757f5d8a] Dart_HandleServiceMessages
  [0x000055d07550e5e0] dart::Isolate::PauseEventHandler()
  [0x000055d0754d79bb] dart::Debugger::Pause(dart::ServiceEvent*)
  [0x000055d0754e13a3] dart::Debugger::PauseException(dart::Instance const&)
  [0x000055d0754f0d7a] dart::Exceptions::Throw(dart::Thread*, dart::Instance const&)
  [0x000055d0754f0d33] Unknown symbol
  [0x000055d0754f0bc6] dart::Exceptions::CreateAndThrowTypeError(dart::TokenPosition, dart::AbstractType const&, dart::AbstractType const&, dart::String const&, dart::String const&)
  [0x000055d07543219e] Unknown symbol
  [0x000055d0754320e3] Unknown symbol
  [0x00007f3170c00bf9] Unknown symbol
  [0x00007f3158588ad0] Unknown symbol
  [0x00007f3170c00ee3] Unknown symbol
  [0x000055d0754cfe05] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000055d0755a7bf5] Unknown symbol
  [0x000055d0755a8108] dart::Instance::InvokeSetter(dart::String const&, dart::Instance const&, bool) const
  [0x000055d07545f98f] dart::BootstrapNatives::DN_InstanceMirror_invokeSetter(_Dart_NativeArguments*)
  [0x00007f3170c00bf9] Unknown symbol
  [0x00007f31589bac51] Unknown symbol
  [0x00007f31589bab98] Unknown symbol
  [0x00007f31589b0020] Unknown symbol
  [0x00007f3158582145] Unknown symbol
  [0x00007f31646ff756] Unknown symbol
  [0x00007f3159090df9] Unknown symbol
  [0x00007f315e63389f] Unknown symbol
  [0x00007f315e633782] Unknown symbol
  [0x00007f31646c9bb9] Unknown symbol
  [0x00007f3159092eb8] Unknown symbol
  [0x00007f31646ff756] Unknown symbol
  [0x00007f315908a8eb] Unknown symbol
  [0x00007f3159092d02] Unknown symbol
  [0x00007f315e62c1e8] Unknown symbol
  [0x00007f315e62bfcd] Unknown symbol
  [0x00007f315e62b905] Unknown symbol
  [0x00007f315e63337b] Unknown symbol
  [0x00007f315e632fc5] Unknown symbol
  [0x00007f315e632099] Unknown symbol
  [0x00007f315deec6ec] Unknown symbol
  [0x00007f315deec22d] Unknown symbol
  [0x00007f315deebb6f] Unknown symbol
  [0x00007f315deeb707] Unknown symbol
  [0x00007f3159091bad] Unknown symbol
  [0x00007f315e63389f] Unknown symbol
  [0x00007f315e633782] Unknown symbol
  [0x00007f31646c9bb9] Unknown symbol
  [0x00007f3159092eb8] Unknown symbol
  [0x00007f31646ff756] Unknown symbol
  [0x00007f315908a8eb] Unknown symbol
  [0x00007f3159092d02] Unknown symbol
  [0x00007f315e62c1e8] Unknown symbol
  [0x00007f315e62bfcd] Unknown symbol
  [0x00007f315e62b905] Unknown symbol
  [0x00007f315e63337b] Unknown symbol
  [0x00007f315e632fc5] Unknown symbol
  [0x00007f315e632099] Unknown symbol
  [0x00007f315e2124e0] Unknown symbol
  [0x00007f31646ff756] Unknown symbol
  [0x00007f315908a8eb] Unknown symbol
  [0x00007f315908a6e7] Unknown symbol
  [0x00007f315e60e4df] Unknown symbol
  [0x00007f315e60e25f] Unknown symbol
  [0x00007f315e60d8a2] Unknown symbol
  [0x00007f315e63093f] Unknown symbol
  [0x00007f315e630800] Unknown symbol
  [0x00007f315e625c89] Unknown symbol
  [0x00007f315e62583f] Unknown symbol
  [0x00007f315e625717] Unknown symbol
  [0x00007f31590896a7] Unknown symbol
  [0x00007f3159087a78] Unknown symbol
  [0x00007f315908765a] Unknown symbol
  [0x00007f31646c9bb9] Unknown symbol
  [0x00007f31646eb470] Unknown symbol
  [0x00007f3170c00ee3] Unknown symbol
  [0x000055d0754cfe05] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000055d0754d3e18] dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
  [0x000055d0755063bb] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000055d07553810a] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000055d07553889c] dart::MessageHandler::TaskCallback()
  [0x000055d075687b6d] dart::ThreadPool::Worker::Loop()
  [0x000055d075687a04] dart::ThreadPool::Worker::Main(unsigned long)
  [0x000055d0755e2cc9] Unknown symbol
-- End of DumpStackTrace

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

[jcollins-g]

I'm wondering if there's some change in the behavior of the Dart json decoder that could explain this, too.

[jcollins-g]

While 2.1.0-dev.7.1 passes the tests, there are still similar debugger crashes when attempting to inspect the same list. Unfortunately this means it is going to be hard for me to figure out what changed just by inspection with the debugger.

Example crash from 2.1.0-dev.7.1:

Testing started at 3:45 PM ...
/usr/local/google/home/jcollins/dart/all_sdks/current/bin/dart --pause_isolates_on_start --enable-vm-service:45125 /usr/local/google/home/jcollins/dart/all_sdks/current/bin/snapshots/pub.dart.snapshot run test -r json /usr/local/google/home/jcollins/dart/rpc/test/api_config_test.dart
Observatory listening on http://127.0.0.1:45125/
../../runtime/platform/utils_linux.cc: 32: error: Fatal error in Utils::VSNPrint with format 'data:application/dart;charset=utf-8,%20%20%20%20import%20%22dart:isolate%22;%0A%0A%20%20%20%20import%20%22package:stream_channel/stream_channel.dart%22;%0A%0A%20%20%20%20import%20%22package:test/src/runner/plugin/remote_platform_helpers.dart%22;%0A%20%20%20%20import%20%22package:test/src/runner/vm/catch_isolate_errors.dart%22;%0A%0A%20%20%20%20import%20%22file:///usr/local/google/home/jcollins/dart/rpc/test/api_config_test.dart%22%20as%20test;%0A%0A%20%20%20%20void%20main(_,%20SendPort%20message)%20%7B%0A%20%20%20%20%20%20var%20channel%20=%20serializeSuite(()%20%7B%0A%20%20%20%20%20%20%20%20catchIsolateErrors();%0A%20%20%20%20%20%20%20%20return%20test.main;%0A%20%20%20%20%20%20%7D);%0A%20%20%20%20%20%20new%20IsolateChannel.connectSend(message).pipe(channel);%0A%20%20%20%20%7D%0A%20%20:1: Error: Can't access platform private library.'
Dumping native stack trace for thread 176bc
  [0x000055aeaefcfb0f] dart::Profiler::DumpStackTrace(void*)
  [0x000055aeaefcfb0f] dart::Profiler::DumpStackTrace(void*)
  [0x000055aeaf220ca2] dart::Assert::Fail(char const*, ...)
  [0x000055aeaf2220e3] Unknown symbol
  [0x000055aeaef04cb2] dart::JSONStream::PrintError(long, char const*, ...)
  [0x000055aeaf03c710] Unknown symbol
  [0x000055aeaf038be8] dart::Service::InvokeMethod(dart::Isolate*, dart::Array const&, bool)
  [0x000055aeaf039375] dart::Service::HandleIsolateMessage(dart::Isolate*, dart::Array const&)
  [0x000055aeaeeeb703] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000055aeaef1cb0e] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000055aeaef1cd67] dart::MessageHandler::HandleOOBMessages()
  [0x000055aeaf1f86fa] Dart_HandleServiceMessages
  [0x000055aeaeef3850] dart::Isolate::PauseEventHandler()
  [0x000055aeaeebd28b] dart::Debugger::Pause(dart::ServiceEvent*)
  [0x000055aeaeecc5ca] dart::Debugger::PauseBreakpoint()
  [0x000055aeaf023930] dart::DRT_BreakpointRuntimeHandler(dart::NativeArguments)
  [0x00007f0fe7ec0a82] [Stub] CallToRuntime
  [0x00007f0fe7ec19c1] [Stub] ICCallBreakpoint
  [0x00007f0fd3822318] ApiConfigSchemaProperty._fromRequest
  [0x00007f0fd380f5a1] ApiConfigSchemaProperty.fromRequest
  [0x00007f0fd380daef] ApiConfigSchema.fromRequest
  [0x00007f0fd38216ba] main.<anonymous closure>.<anonymous closure>
  [0x00007f0fcdb43538] _Closure.call
  [0x00007f0fd4624b7b] Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure>
  [0x00007f0fcdb799a1] _Closure.call
  [0x00007f0fcdb7987c] _asyncThenWrapperHelper.<anonymous closure>
  [0x00007f0fd8fc0dfb] _Closure.call
  [0x00007f0fd4626d52] StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure>
  [0x00007f0fcdb43538] _Closure.call
  [0x00007f0fd461de97] StackZoneSpecification._run
  [0x00007f0fd4626b81] StackZoneSpecification._registerUnaryCallback.<anonymous closure>
  [0x00007f0fcdb71ed8] _rootRunUnary
  [0x00007f0fcdb71c96] _rootRunUnary
  [0x00007f0fcdb71612] _CustomZone.runUnary
  [0x00007f0fcdb79449] _FutureListener.handleValue
  [0x00007f0fcdb790a2] _Future._propagateToListeners.handleValueCallback
  [0x00007f0fcdb78058] _Future._propagateToListeners
  [0x00007f0fcd5bc466] _Future._complete
  [0x00007f0fcd5bbfbd] _SyncCompleter.complete
  [0x00007f0fcd5bb931] _AsyncAwaitCompleter.complete
  [0x00007f0fcd5bb553] _completeOnAsyncReturn
  [0x00007f0fd4625a39] Declarer._runSetUps
  [0x00007f0fcdb799a1] _Closure.call
  [0x00007f0fcdb7987c] _asyncThenWrapperHelper.<anonymous closure>
  [0x00007f0fd8fc0dfb] _Closure.call
  [0x00007f0fd4626d52] StackZoneSpecification._registerUnaryCallback.<anonymous closure>.<anonymous closure>
  [0x00007f0fcdb43538] _Closure.call
  [0x00007f0fd461de97] StackZoneSpecification._run
  [0x00007f0fd4626b81] StackZoneSpecification._registerUnaryCallback.<anonymous closure>
  [0x00007f0fcdb71ed8] _rootRunUnary
  [0x00007f0fcdb71c96] _rootRunUnary
  [0x00007f0fcdb71612] _CustomZone.runUnary
  [0x00007f0fcdb79449] _FutureListener.handleValue
  [0x00007f0fcdb790a2] _Future._propagateToListeners.handleValueCallback
  [0x00007f0fcdb78058] _Future._propagateToListeners
  [0x00007f0fcd85953a] _Future._addListener.<anonymous closure>
  [0x00007f0fcdb43538] _Closure.call
  [0x00007f0fd461de97] StackZoneSpecification._run
  [0x00007f0fd461dc47] StackZoneSpecification._registerCallback.<anonymous closure>
  [0x00007f0fcdb5316a] _rootRun
  [0x00007f0fcdb52e85] _rootRun
  [0x00007f0fcdb524cf] _CustomZone.run
  [0x00007f0fcdb76793] _CustomZone.runGuarded
  [0x00007f0fcdb7663a] _CustomZone.bindCallbackGuarded.<anonymous closure>
  [0x00007f0fcdb6b8c3] _microtaskLoop
  [0x00007f0fcdb6b3dd] _startMicrotaskLoop
  [0x00007f0fcdb6b279] _startMicrotaskLoop
  [0x00007f0fd461cba0] _Timer._runTimers
  [0x00007f0fd461ad3b] _Timer._handleMessage
  [0x00007f0fd461a87c] _Timer._handleMessage
  [0x00007f0fd8fc0dfb] _Closure.call
  [0x00007f0fd8fee102] _RawReceivePortImpl._handleMessage
  [0x00007f0fe7ec0ee3] [Stub] InvokeDartCode
  [0x000055aeaeeb58a5] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000055aeaeeb98b8] dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
  [0x000055aeaeeeb51b] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000055aeaef1ca3a] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000055aeaef1d1cc] dart::MessageHandler::TaskCallback()
  [0x000055aeaf06ef5d] dart::ThreadPool::Worker::Loop()
  [0x000055aeaf06edf4] dart::ThreadPool::Worker::Main(unsigned long)
  [0x000055aeaefcbff9] Unknown symbol
-- End of DumpStackTrace

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

[rmacnak-google]

At least the second order bug: https://dart-review.googlesource.com/c/sdk/+/82460

[leafpetersen]

  List _fromRequest(List encodedList) {
-    return encodedList.map(_itemsProperty._fromRequest).toList();
+    List result = encodedList.map(_itemsProperty._fromRequest).toList();
+    if (result.every((dynamic value) => value is Map)) {
+      return new List<Map>.from(result);
+    } else if (result.every((dynamic value) => value is List)) {
+      return new List<List>.from(result);
+    } else if (result.every((dynamic value) => value is String)) {
+      return new List<String>.from(result);
+    }
+    return result;
   }
 }

I suspect this is a bandaid around some other issue (possibly a change in other APIs?). I looked at the RPC code, and the semantics of the pre-patch code is definitely not the same as the semantics of the post-patch code, unless encodedList is an instance of your own list class that is doing something special in .map to set the reified type. But if encodedList is a standard list (or something with standard list type semantics), the result of the .map call will be a List<dynamic> in the original code.

[jcollins-g]

@leafpetersen Yes, that patch is completely a band-aid around something else going on. Exactly what is going on is hard to tell due to the VM crashes when I am poking around in the debugger; maybe a651838 will fix that. @rmacnak-google Can you have that merged to stable for 2.1? The debugger crashed an awful lot with this problem.

It's still a blocker that types aren't coming out of those methods like they used to in 7.1, it's just the problems with the debugger made me resort to strange contortions like that patch and printf to read the tea leaves as to what changed. Will take a look when I'm feeling better (out sick today), or if someone else would like to try I would appreciate the support.

[mit-mit]

cc @rmacnak-google @a-siva any news about this one? This sounds like an unwanted breaking change?

[kmillikin]

It bisects to this CL: https://dart-review.googlesource.com/c/sdk/+/79081

[a-siva]

The change listed above introduces strong mode semantics when dart:mirrors API is used. Does the rpc package use dart:mirrors?

[a-siva]

/cc @bkonyi

[bkonyi]

It looks like in invokeWithBody in config/method.dart an exception is being thrown. Here's the exception for the api-invoke-post reverse-list test:

invocation(symbol: Symbol("reverseListPost"), pos: [[1, 2, 3]], named: {})
CAUGHT: type 'List<dynamic>' is not a subtype of type 'List<String>' of 'request'

As you can see, the positional arguments don't match the type of reverseListPost. This was valid before since we weren't correctly performing type checks for invocations make using mirrors, which was fixed in the CL @kmillikin found during his bisect.

[jcollins-g]

@a-siva Yes, the RPC package makes extensive use of mirrors.

[jcollins-g]

@bkonyi That makes sense. I'm kind of surprised we're the only one bumping into this, but it might just be a quirk of how the package is implemented. Given that the debugger is fixed at head, I should have a shot at working out a solution if one is possible without extensive client modifications. I'll work on that ASAP.

[jcollins-g]

Now that I have the debugger (thanks!) I'm understanding a lot more how we get into the situation that @bkonyi refers to in the rpc package. At a high level, the rpc package is doing some tasks similar to the protocol buffer compiler. Except where the protocol buffer compiler uses generated source code files, rpc is using mirrors and class metadata. This runs into limitations in a world of strong-mode mirrors.

One example of the dilemma:

https://github.com/dart-lang/rpc/blob/5ad1009b1c4ec5771cef7c67de74ee43328e164a/lib/src/parser.dart#L1013

What we really need to do is be able to pass an appropriate type parameter to the ListProperty constructor (so we eventually get List<String> instead of List<dynamic>), but we can't vivify a type parameter to pass from the mirror because we can't handle types in the same way as other objects in Dart. This all worked before precisely because there were no strong mode checks in mirrors.

I'm pretty sure we're toast in the general case unless we do one of the following:

1. Stop using mirrors in a way that results in us constructing objects that are supposed to be compatible with the mirrored types. (That would break the rpc package for many non-trivial and some trivial use cases.) I believe I could hack up DartPad and/or rpc to make that combination work for 2.1, but any other non-trivial users will be forced to migrate.
2. We revert SDK CL https://dart-review.googlesource.com/c/sdk/+/79081 and maybe try again to make this breaking change in 2.2 -- might be viable if the rpc package really is the only one that cares and we successfully turn it down by then.
3. We extend the language and/or the mirrors library to make it possible to vivify working type parameters from mirrors. This doesn't seem to be the direction Dart is headed, but would technically work.

If I'm right here it's my opinion #2 is safest, and we can have more time to get clients off the rpc package. If I understand right @mit-mit is planning to turn it down soon.

[leafpetersen]

A couple of comments:

• Would this really require language changes? Mirrors is already outside of the language, what stops it from just supporting this use case? It's not clear to me why instantiating a class reflectively is supportable but instantiating a generic class reflectively is not, but then I don't know much about mirrors.
• There is some limited support for this in the language now for Map and List if you have an existing instance, but it's an experimental stopgap measure and not really encouraged: https://pub.dartlang.org/packages/dart_internal
• We have considered adding language based support for functionality like dart_internal, and could move it up the priority queue, but that would still be a ways out, and again, it really would require an instance to work from (which it's not clear to me that you have here).

[jcollins-g]

@leafpetersen I'm not aware of anything preventing it from being incorporated in the mirrors library, as long as we could pass some parameters to an API and have it instantiate a user-defined class with the type parameters from a mirror. I hedged there only due to lack of knowledge.

Unfortunately we're not starting from an existing instance, so I don't think dart_internal based options will work.

[a-siva]

The general idea was that 'dart:mirrors' would be supported to the extent it was used in google3 for the testing infrastructure. Flutter itself does not use 'dart:mirrors'. Somebody pointed out that invocations using mirrors was not doing the correct type checks and that was fixed recently.
Enhancing 'dart:mirrors' to enable all potential use scenarios would be a larger project and I am not sure it is the right investment of our time.

[jcollins-g]

@a-siva Seems reasonable. I doubted that extending dart:mirrors would be the path we'd take.

I do still think that reverting the breaking change and making that a change for 2.2 to give clients time to migrate is the best option here.

[athomas]

This is a blocker for the first 2.2 dev release. The revert was cherry-picked into the 2.1 release branch but the change was then re-landed on master. The re-land wasn't cherry-picked into 2.1, so 2.1 is fine as far as this issue is concerned.

[a-siva]

Yes that was the intent, we want this issue resolved before the next release as not doing type checks is not a good option.

[a-siva]

Assigning issue to mit to figure out how we want to proceed on this

• deprecate use of this rpc package
• rework this package to stop using 'dart:mirrors'
• enhance 'dart:mirrors' to support this use case

[jcollins-g]

@rmacnak-google You've given me some ideas for new exploration... I didn't connect that I could use reflectees on the list of TypeMirrors to create a ClassMirror and then call newInstance on it until I saw your example -- that could conceivably be enough to patch this up. I'll get back to you on that, but in the meantime I'm glad the behavior has been reverted to buy some time.

In the meantime, here is an example of more debugger crashiness while exploring mirrors (SDK at head yesterday, before the revert of the type checking). Modified parseListProperty to be the following, with a breakpoint on the return and I tried to have the debugger evaluate the expression at that line.

  ListProperty parseListProperty(String propertyName, ApiProperty metadata,
      ClassMirror listPropertyType, bool isRequest) {
    // If List<T> is a superclass, the way to get T is different.
    var listTypeArguments;
    if (listPropertyType.originalDeclaration != reflectClass(List)) {
      listTypeArguments = listPropertyType.superinterfaces
          .firstWhere((interface) =>
              interface.originalDeclaration == reflectClass(List))
          .typeArguments;
    } else {
      listTypeArguments = listPropertyType.typeArguments;
    }
    assert(listTypeArguments.length == 1);
    assert(metadata != null);
    var listTypeName = MirrorSystem.getName(listTypeArguments[0].simpleName);
    _checkValidFields(propertyName, 'List<$listTypeName>', metadata, []);
    // TODO: Figure out what to do about metadata for the items property.
    var listItemsProperty = parseProperty(
        listTypeArguments[0], propertyName, new ApiProperty(), isRequest);

    ClassMirror listProperty = reflectType(ListProperty, listTypeArguments.map<Type>((TypeMirror f) => f.reflectedType).toList());
    return listProperty.newInstance(const Symbol(''), [propertyName, metadata.description, metadata.required, listItemsProperty]).reflectee;
    //return new ListProperty(propertyName, metadata.description,
    //    metadata.required, listItemsProperty);
  }

Testing started at 9:53 AM ...
/usr/local/google/home/jcollins/dart/all_sdks/current/bin/dart --pause_isolates_on_start --enable-vm-service:39605 /usr/local/google/home/jcollins/dart/all_sdks/current/bin/snapshots/pub.dart.snapshot run test -r json /usr/local/google/home/jcollins/dart/rpc/test/api_config_test.dart -N request-parsing
Observatory listening on http://127.0.0.1:39605/

===== CRASH =====
version=2.1.0-edge.e58560548881093042a59faba2e074af44d63f11 (Tue Nov 6 17:43:40 2018 +0000) on "linux_x64"
si_signo=Segmentation fault(11), si_code=2, si_addr=0x7f7ea0ec0041
Dumping native stack trace for thread 3a8b7
  [0x00007f7ea0ec0041] Unknown symbol
  [0x00007f7ea0ec0041] Unknown symbol
  [0x000055c8f9fab955] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000055c8fa236e1f] dart::kernel::ConstantEvaluator::RunFunction(dart::TokenPosition, dart::Function const&, dart::Array const&, dart::Array const&)
  [0x000055c8fa236cc6] dart::kernel::ConstantEvaluator::RunFunction(dart::TokenPosition, dart::Function const&, long, dart::Instance const*, dart::TypeArguments const*)
  [0x000055c8fa2335a0] dart::kernel::ConstantEvaluator::EvaluateStaticInvocation()
  [0x000055c8fa23624c] dart::kernel::ConstantEvaluator::EvaluateStaticInvocation(long, bool)
  [0x000055c8fa248826] dart::kernel::StreamingFlowGraphBuilder::BuildStaticInvocation(bool, dart::TokenPosition*)
  [0x000055c8fa23b38e] dart::kernel::StreamingFlowGraphBuilder::BuildExpression(dart::TokenPosition*)
  [0x000055c8fa25133a] dart::kernel::StreamingFlowGraphBuilder::BuildArgumentsFromActualArguments(dart::Array*, bool, bool)
  [0x000055c8fa2472d7] dart::kernel::StreamingFlowGraphBuilder::BuildMethodInvocation(dart::TokenPosition*)
  [0x000055c8fa23b36c] dart::kernel::StreamingFlowGraphBuilder::BuildExpression(dart::TokenPosition*)
  [0x000055c8fa2447c3] dart::kernel::StreamingFlowGraphBuilder::BuildPropertyGet(dart::TokenPosition*)
  [0x000055c8fa23b0fd] dart::kernel::StreamingFlowGraphBuilder::BuildExpression(dart::TokenPosition*)
  [0x000055c8fa24e8b0] dart::kernel::StreamingFlowGraphBuilder::BuildReturnStatement()
  [0x000055c8fa23d39e] dart::kernel::StreamingFlowGraphBuilder::BuildStatement()
  [0x000055c8fa242c03] dart::kernel::StreamingFlowGraphBuilder::BuildFunctionBody(dart::Function const&, dart::LocalVariable*, bool)
  [0x000055c8fa243902] dart::kernel::StreamingFlowGraphBuilder::BuildGraphOfFunction(bool)
  [0x000055c8fa244107] dart::kernel::StreamingFlowGraphBuilder::BuildGraph()
  [0x000055c8fa259149] dart::kernel::FlowGraphBuilder::BuildGraph()
  [0x000055c8fa28ca3a] dart::DartCompilationPipeline::BuildFlowGraph(dart::Zone*, dart::ParsedFunction*, dart::ZoneGrowableArray<dart::ICData const*>*, long, bool)
  [0x000055c8fa28f7e1] dart::CompileParsedFunctionHelper::Compile(dart::CompilationPipeline*)
  [0x000055c8fa290820] Unknown symbol
  [0x000055c8fa290198] dart::Compiler::CompileFunction(dart::Thread*, dart::Function const&)
  [0x000055c8f9fab79d] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000055c8fa03cd0d] Unknown symbol
  [0x000055c8f9fb8161] dart::ActivationFrame::EvaluateCompiledExpression(unsigned char const*, long, dart::Array const&, dart::Array const&, dart::TypeArguments const&)
  [0x000055c8fa131013] Unknown symbol
  [0x000055c8fa126be8] dart::Service::InvokeMethod(dart::Isolate*, dart::Array const&, bool)
  [0x000055c8fa127375] dart::Service::HandleIsolateMessage(dart::Isolate*, dart::Array const&)
  [0x000055c8f9fe01e3] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000055c8fa011efe] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000055c8fa012157] dart::MessageHandler::HandleOOBMessages()
  [0x000055c8fa2ca84a] Dart_HandleServiceMessages
  [0x000055c8f9fe79f0] dart::Isolate::PauseEventHandler()
  [0x000055c8f9fb281b] dart::Debugger::Pause(dart::ServiceEvent*)
  [0x000055c8f9fc105a] dart::Debugger::PauseBreakpoint()
  [0x000055c8fa111910] dart::DRT_BreakpointRuntimeHandler(dart::NativeArguments)
  [0x00007f7ea0e40f02] Unknown symbol
  [0x00007f7ea0e41e41] Unknown symbol
  [0x00007f7e8ebf25ce] Unknown symbol
  [0x00007f7e8ebe9596] Unknown symbol
  [0x00007f7e8ebe6230] Unknown symbol
  [0x00007f7e8ebe5a9c] Unknown symbol
  [0x00007f7e8ebe5584] Unknown symbol
  [0x00007f7e8ebd959a] Unknown symbol
  [0x00007f7e8ebd3cb6] Unknown symbol
  [0x00007f7e9437f4b6] Unknown symbol
  [0x00007f7e8ebcfe79] Unknown symbol
  [0x00007f7e87af317f] Unknown symbol
  [0x00007f7e87af3062] Unknown symbol
  [0x00007f7e943498b9] Unknown symbol
  [0x00007f7e8ebd1eb8] Unknown symbol
  [0x00007f7e9437f4b6] Unknown symbol
  [0x00007f7e8ebc994b] Unknown symbol
  [0x00007f7e8ebd1cfe] Unknown symbol
  [0x00007f7e87aebaa8] Unknown symbol
  [0x00007f7e87aeb895] Unknown symbol
  [0x00007f7e87aeb1b6] Unknown symbol
  [0x00007f7e87af2c66] Unknown symbol
  [0x00007f7e87af28a5] Unknown symbol
  [0x00007f7e87af1979] Unknown symbol
  [0x00007f7e8732d5ac] Unknown symbol
  [0x00007f7e8732cbe5] Unknown symbol
  [0x00007f7e8732c527] Unknown symbol
  [0x00007f7e8732c0c7] Unknown symbol
  [0x00007f7e8ebd0c2d] Unknown symbol
  [0x00007f7e87af317f] Unknown symbol
  [0x00007f7e87af3062] Unknown symbol
  [0x00007f7e943498b9] Unknown symbol
  [0x00007f7e8ebd1eb8] Unknown symbol
  [0x00007f7e9437f4b6] Unknown symbol
  [0x00007f7e8ebc994b] Unknown symbol
  [0x00007f7e8ebd1cfe] Unknown symbol
  [0x00007f7e87aebaa8] Unknown symbol
  [0x00007f7e87aeb895] Unknown symbol
  [0x00007f7e87aeb1b6] Unknown symbol
  [0x00007f7e87af2c66] Unknown symbol
  [0x00007f7e87af28a5] Unknown symbol
  [0x00007f7e87af1979] Unknown symbol
  [0x00007f7e876d0f80] Unknown symbol
  [0x00007f7e9437f4b6] Unknown symbol
  [0x00007f7e8ebc994b] Unknown symbol
  [0x00007f7e8ebc973e] Unknown symbol
  [0x00007f7e87ace07f] Unknown symbol
  [0x00007f7e87acddff] Unknown symbol
  [0x00007f7e87acd453] Unknown symbol
  [0x00007f7e87af01ff] Unknown symbol
  [0x00007f7e87af00c0] Unknown symbol
  [0x00007f7e87ae5589] Unknown symbol
  [0x00007f7e87ae513f] Unknown symbol
  [0x00007f7e87ae5017] Unknown symbol
  [0x00007f7e8ebc87e7] Unknown symbol
  [0x00007f7e8ebc6bb8] Unknown symbol
  [0x00007f7e8ebc679a] Unknown symbol
  [0x00007f7e943498b9] Unknown symbol
  [0x00007f7e9436b250] Unknown symbol
  [0x00007f7ea0e41363] Unknown symbol
  [0x000055c8f9fab955] dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  [0x000055c8f9faedb0] dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
  [0x000055c8f9fdfffb] dart::IsolateMessageHandler::HandleMessage(dart::Message*)
  [0x000055c8fa011e2a] dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  [0x000055c8fa0125bc] dart::MessageHandler::TaskCallback()
  [0x000055c8fa15f10d] dart::ThreadPool::Worker::Loop()
  [0x000055c8fa15efa4] dart::ThreadPool::Worker::Main(unsigned long)
  [0x000055c8fa0ba6f9] Unknown symbol
-- End of DumpStackTrace

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

[a-siva]

@jcollins-g I am going to open a new issue for the debugger exception that you saw so we can track that separately from the above 'dart:mirrors' issue.

[jcollins-g]

Update: it now looks possible to convert the rpc package. I've completed an ugly proof-of-concept conversion (only for the List type), here: https://github.com/dart-lang/rpc/compare/experimental-mirrors-refactor. It won't be a quick fix, but there's enough support in the mirrors library to make it happen.

[mit-mit]

@jcollins-g any news on the rpc package?

[jcollins-g]

@mit-mit I think it's possible to close this out now for 2.1. The rpc package will be broken for 2.2 but there's a pathway out via a large-scale refactor (probably a week or so of intense hacking + testing time).

[mit-mit]

Is there a second bug tracking fixing it in 2.2? I though the present bug was tracking that now...

[jcollins-g]

dart-archive/rpc#133 is now tracking the issue for the rpc package (and I just updated it with the latest status), but if you prefer it to be tracked here as well we can repurpose this. It probably should be out of area-vm, assuming we're satisfied with requiring the rpc package to be changed for 2.2.

[Pareshoct7]

Still issue is not resolved I tried with dev channel also.

here is the error:

Steps to Reproduce

flutter build ios --release

Logs

Running pod install...                                      22.6s
Starting Xcode build...                                          
 ├─Building Dart code...                             11.3s                                                                                                                                                                                                           
Xcode build done.                                           21.0s
Failed to build iOS app
Error output from Xcode build:
↳
    ** BUILD FAILED **


Xcode's output:
↳
    === BUILD TARGET Runner OF PROJECT Runner WITH CONFIGURATION Release ===
    The use of Swift 3 @objc inference in Swift 4 mode is deprecated. Please address deprecated @objc inference warnings, test your code with “Use of deprecated Swift 3 @objc inference” logging enabled, and then disable inference by changing the "Swift 3 @objc
    Inference" build setting to "Default" for the "Runner" target.
    The use of Swift 3 @objc inference in Swift 4 mode is deprecated. Please address deprecated @objc inference warnings, test your code with “Use of deprecated Swift 3 @objc inference” logging enabled, and then disable inference by changing the "Swift 3 @objc
    Inference" build setting to "Default" for the "Runner" target.
    === BUILD TARGET Runner OF PROJECT Runner WITH CONFIGURATION Release ===
    Building AOT snapshot in release mode (ios-release)...          

    Compiler message:
    lib/screens/ProfileViewScreen.dart:1:8: Error: Can't access platform private library.
    import 'dart:_http';
           ^
    lib/screens/swipe/cards_section_draggable.dart:1:8: Error: Can't access platform private library.
    import 'dart:_http';
           ^
    Compiler terminated unexpectedly.
    Failed to build /Users/pareshpatil/workspace/kaledate/mobile.

Encountered error while building for device. 

flutter doctor -v

[✓] Flutter (Channel dev, v0.11.9, on Mac OS X 10.13.6 17G65, locale en-IN)
    • Flutter version 0.11.9 at /Users/pareshpatil/sdk/flutter
    • Framework revision d48e6e433c (2 days ago), 2018-11-20 22:05:23 -0500
    • Engine revision 5c8147450d
    • Dart version 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)

[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
    • Android SDK at /Users/pareshpatil/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.1, Build version 10B61
    • ios-deploy 2.0.0
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 29.1.1
    • Dart plugin version 181.5656
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1136-b06)

[✓] IntelliJ IDEA Community Edition (version 2018.2.6)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 30.0.2
    • Dart plugin version 182.5124

[✓] Connected device (1 available)
    • iPhone XR • 21381D8F-7EEA-4E74-91AE-1302FC4615D2 • ios • iOS 12.1 (simulator)

• No issues found!

[mraleph]

@Pareshoct7 your issue has no connection to this bug.

[athomas]

@jcollins-g I'll close this, feel free to re-open if the issue still exists.