Optional API parameter bug

[abitmore]

Bug Description

If all parameters of an API are optional, calling the API with no parameter will crash the node. Fortunately we don't have such API in production.

A sample API:

bitshares-core/libraries/app/include/graphene/app/database_api.hpp

Lines 407 to 409 in c5e8585

	vector<general_asset_info> list_assets_general_info( optional<string> lower_bound_symbol = {},
	optional<uint32_t> limit = {},
	optional<general_asset_info::asset_type> type = {} )const;

Stack back trace:

#0 0x00000000010ee5b0 in fc::variant::get_type() const ()
#1 0x00000000010ee799 in fc::variant::is_null() const ()
#2 0x0000000000b9451b in std::vector<graphene::app::general_asset_info, std::allocatorgraphene::app::general_asset_info > fc::generic_api::call_generic<std::vector<graphene::app::general_asset_info, std::allocatorgraphene::app::general_asset_info >, fc::optional<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, fc::optional, fc::optionalgraphene::app::general_asset_info::asset_type >(std::function<std::vector<graphene::app::general_asset_info, std::allocatorgraphene::app::general_asset_info > (fc::optional<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, fc::optional, fc::optionalgraphene::app::general_asset_info::asset_type)> const&, __gnu_cxx::__normal_iterator<fc::variant const*, std::vector<fc::variant, std::allocatorfc::variant > >, __gnu_cxx::__normal_iterator<fc::variant const*, std::vector<fc::variant, std::allocatorfc::variant > >, unsigned int) ()
#3 0x0000000000b955aa in std::_Function_handler<fc::variant (std::vector<fc::variant, std::allocatorfc::variant > const&), std::function<fc::variant (std::vector<fc::variant, std::allocatorfc::variant > const&)> fc::generic_api::api_visitor::to_generic<std::vector<graphene::app::general_asset_info, std::allocatorgraphene::app::general_asset_info >, fc::optional<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, fc::optional, fc::optionalgraphene::app::general_asset_info::asset_type >(std::function<std::vector<graphene::app::general_asset_info, std::allocatorgraphene::app::general_asset_info > (fc::optional<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, fc::optional, fc::optionalgraphene::app::general_asset_info::asset_type)> const&) const::{lambda(std::vector<fc::variant, std::allocatorfc::variant > const&)#1}>::_M_invoke(std::_Any_data const&, std::vector<fc::variant, std::allocatorfc::variant > const&) ()
#4 0x000000000116f541 in fc::generic_api::call(unsigned int, std::vector<fc::variant, std::allocatorfc::variant > const&) ()
#5 0x000000000116f8ac in fc::generic_api::call(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<fc::variant, std::allocatorfc::variant > const&) ()
#6 0x000000000116fb7a in fc::api_connection::receive_call(unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<fc::variant, std::allocatorfc::variant > const&) const ()
#7 0x0000000001169f2c in std::_Function_handler<fc::variant (std::vector<fc::variant, std::allocatorfc::variant > const&), fc::rpc::websocket_api_connection::websocket_api_connection(std::shared_ptrfc::http::websocket_connection const&, unsigned int)::{lambda(std::vector<fc::variant, std::allocatorfc::variant > const&)#1}>::_M_invoke(std::_Any_data const&, std::vector<fc::variant, std::allocatorfc::variant > const&) ()
#8 0x00000000012261d4 in fc::rpc::state::local_call(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::vector<fc::variant, std::allocatorfc::variant > const&) ()
#9 0x000000000116b636 in fc::rpc::websocket_api_connection::on_request(fc::variant const&) ()
#10 0x000000000116d1d4 in fc::rpc::websocket_api_connection::on_message(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) ()
#11 0x000000000116ddea in std::_Function_handler<fc::http::reply (std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&), fc::rpc::websocket_api_connection::websocket_api_connection(std::shared_ptrfc::http::websocket_connection const&, unsigned int)::{lambda(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)#6}>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) ()
#12 0x0000000001215d13 in fc::http::detail::websocket_server_impl::websocket_server_impl()::{lambda(std::weak_ptr)#4}::operator()(std::weak_ptr) const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const ()
#13 0x0000000001216439 in fc::detail::void_functor_run<fc::http::detail::websocket_server_impl::websocket_server_impl()::{lambda(std::weak_ptr)#4}::operator()(std::weak_ptr) const::{lambda()#1}::operator()() const::{lambda()#1}>::run(void*, {lambda()#1}) ()
#14 0x0000000001113eb4 in fc::task_base::run_impl() ()
#15 0x000000000111236f in fc::thread_d::process_tasks() ()
#16 0x0000000001112b1c in fc::thread_d::start_process_tasks(long) ()
#17 0x0000000001853e61 in make_fcontext ()
#18 0x0000000000000000 in ?? ()

Impacts
Describe which portion(s) of BitShares Core may be impacted by this bug. Please tick at least one box.

• API (the application programming interface)
• Build (the build process or something prior to compiled code)
• CLI (the command line wallet)
• Deployment (the deployment process after building such as Docker, Travis, etc.)
• DEX (the Decentralized EXchange, market engine, etc.)
• P2P (the peer-to-peer network for transaction/block propagation)
• Performance (system or user efficiency, etc.)
• Protocol (the blockchain logic, consensus, validation, etc.)
• Security (the security of system or user data, etc.)
• UX (the User Experience)
• Other (please add below)

Steps To Reproduce
Steps to reproduce the behavior (example outlined below):

1. Execute API call '...'
2. Using JSON payload '...'
3. Received response '...'
4. See error in screenshot

Expected Behavior
A clear and concise description of what you expected to happen.

Screenshots (optional)
If applicable, add screenshots to help explain process flow and behavior.

Host Environment
Please provide details about the host environment. Much of this information can be found running: witness_node --version.

• Host OS: [e.g. Ubuntu 18.04 LTS]
• Host Physical RAM [e.g. 4GB]
• BitShares Version: [e.g. 2.0.180425]
• OpenSSL Version: [e.g. 1.1.0g]
• Boost Version: [e.g. 1.65.1]

Additional Context (optional)
Add any other context about the problem here.

CORE TEAM TASK LIST

• Evaluate / Prioritize Bug Report
• Refine User Stories / Requirements
• Define Test Cases
• Design / Develop Solution
• Perform QA/Testing
• Update Documentation

[abitmore]

And something which I don't know if it's the same bug:

The first argument of the API is optional<string>. Sometimes this API call with an empty string will return an error.

$ curl -d '{"id":1,"method":"call","params":["database","list_assets_general_info",[""]]}' http://localhost:7788/ 2>/dev/null;echo
{"id":1,"error":{"code":7,"message":"Execution error","data":{"code":7,"name":"bad_cast_exception","message":"Bad Cast","stack":[{"context":{"level":"error","file":"variant.cpp","line":389,"method":"as_int64","hostname":"","thread_name":"th_a","timestamp":"2019-08-10T22:03:34"},"format":"Invalid cast from ${type} to int64","data":{"type":"101"}}]}}}

Unable to reproduce stably.

[nathanielhourt]

Unable to reproduce in unit tests... I'll attempt to reproduce in an actual BitShares API tomorrow...

In the meantime, my first guess from looking at the code is that this is caused by this line which has an a0+1 when a0 is already in the past-the-end-iterator state (also on line 53) but I'll be unable to test if that fixes it until I can reproduce the crash.

[abitmore]

To reproduce, build the pr-general-info-api branch and start a node with --seed-nodes="[]", no need to download chain data.

[nathanielhourt]

Crash reproduced and fixed. I was unable to reproduce the empty string error.

[oxarbitrage]

Fixed by bitshares/bitshares-fc#146 but need to bump FC to close.

[ryanRfox]

Done. Bumped FC in #1912.