Dynamic arguments storage. Implementation of enhancement from issue #1170. #1584

vsolontsov-ll · 2020-03-09T17:55:33Z

I agree that my contributions are licensed under the {fmt} license, and agree to future changes to the licensing.

(Hope I didn't miss any guidelines.)

fmtlib#1170. First try...

vsolontsov-ll · 2020-03-10T08:38:57Z

Travis builds are fixed. Let's give a try with appveyor (don't know how to configure it for my fork yet)

vitaut

Thanks for the PR! Please address the initial comments (inline).

include/fmt/core.h

include/fmt/dyn-args.h

named_args will be added in a separate PR. Pending: avoid Args types from dynamic_format_arg_store template parameters, replace std::forward_list<std::variant> with a custom list.

Pending: cleanup.

Removed constexpr

Pending -- move dyn-args.h to core.h

vsolontsov-ll · 2020-03-14T10:03:21Z

Maybe worth adding

dynamic_format_arg_store::reserve(size_t)
static_assert into void push_back(std::reference_wrapper<T> arg) to prevent passing primitive types by ref

But I'd leave those for the a PR with named args.

vitaut

Looks great. I left a few more comments, mostly minor/nits.

vitaut · 2020-03-14T13:24:27Z

include/fmt/core.h

+#ifdef FMT_USE_STRING_VIEW
+template <typename Traits, typename Char>
+struct is_string_view<std::basic_string_view<Char, Traits>, Char>
+    : std::true_type {};
+#endif
+
+#ifdef FMT_USE_EXPERIMENTAL_STRING_VIEW
+template <typename Traits, typename Char>
+struct is_string_view<std::experimental::basic_string_view<Char, Traits>, Char>
+    : std::true_type {};
+#endif


This can be replaced with

template <typename Char> struct is_string_view<std_string_view<Char>, Char> : std::true_type {};

I wouldn't bother with traits at this time since they are not handled elsewhere in {fmt}.

Well, I'd create an issue to track it then...
There's a constructor for internal::basic_string_view from std::basic_string<> with all the template parameters. But std_string_view is recognized with only defaults...

vitaut · 2020-03-14T13:24:35Z

include/fmt/core.h

+template <typename T> struct is_ref_wrapper : std::false_type {};
+
+template <typename T>
+struct is_ref_wrapper<std::reference_wrapper<T>> : std::true_type {};


Why do we need to handle std::reference_wrapper?

Hm, for me it's the main use-case :- ).
It's to let one to avoid dynamic memory allocation when the argument lifetime is guaranteed.

Actually I would want to make the dynamic_format_arg_store reusable and more friendly for those who concerned about memory allocations:

add reserve(); -- reserves

add clean(); -- drops the storage_ and resizes the data_ to zero, but doesn't de-allocate the storage_.

Makes sense. Thanks for clarification.

Also let's rename this trait to is_reference_wrapper for consistency with the class name itself.

vitaut · 2020-03-14T13:28:17Z

include/fmt/core.h

+struct is_ref_wrapper<std::reference_wrapper<T>> : std::true_type {};
+
+template <typename T, typename Context> struct need_dyn_copy {
+  using mapped_type = mapped_type_constant<T, Context>;


Since you only use mapped_type::value I suggest replacing this with

static internal::type mapped_type = mapped_type_constant<T, Context>::value;

and using mapped_type in place of mapped_type::value.

vitaut · 2020-03-14T13:29:20Z

include/fmt/core.h

+template <typename T, typename Context>
+using need_dyn_copy_t = typename need_dyn_copy<T, Context>::type;


Let's kill this alias since need_dyn_copy is only used in one place.

vitaut · 2020-03-14T13:29:40Z

include/fmt/core.h

+class dyn_arg_storage {
+  // Workaround clang's -Wweak-vtables. For templates (unlike regular classes
+  // doesn't complain about inability to deduce translation unit to place vtable
+  // So dyn_arg_node_base is made a fake template


Did you mean storage_node_base instead of dyn_arg_node_base?

vitaut · 2020-03-14T13:35:14Z

include/fmt/core.h

+template <typename T, typename Context>
+using need_dyn_copy_t = typename need_dyn_copy<T, Context>::type;
+
+class dyn_arg_storage {


I'd merge this class into dynamic_format_arg_store unless there is a strong reason to have it separate.

vitaut · 2020-03-14T13:37:37Z

include/fmt/core.h

+  using string_type = std::basic_string<char_type>;
+  using value_type = basic_format_arg<Context>;


I'd kill these two aliases since they are only used once each (and the notion of value_type is somewhat unclear here).

vitaut · 2020-03-14T13:38:36Z

include/fmt/core.h

+  // Storage of basic_format_arg must be contiguous
+  // Required by basic_format_args::args_ which is just a pointer


nit: missing punctuation at the end of sentenses (.)

vitaut · 2020-03-14T13:44:11Z

include/fmt/core.h

+  template <typename T> const T& stored_value(const T& arg, std::false_type) {
+    return arg;
+  }
+
+  template <typename T>
+  const T& stored_value(const std::reference_wrapper<T>& arg, std::false_type) {
+    return arg.get();
+  }
+
+  template <typename T>
+  const stored_type<T>& stored_value(const T& arg, std::true_type) {
+    return storage_.push<stored_type<T>>(arg);
+  }


I suggest replacing tag dispatch with a normal if in push_back.

vitaut · 2020-03-14T13:50:25Z

include/fmt/core.h

+    static_assert(
+        !std::is_base_of<internal::named_arg_base<char_type>, T>::value,
+        "Named arguments are not supported yet");
+    emplace_arg(stored_value(arg, internal::need_dyn_copy_t<T, Context>{}));


i.e. replacing this with something like

if (const_check(internal::need_dyn_copy_t<T, Context>())) emplace_arg(storage_.push<stored_type<T>>(arg)); else emplace_arg(arg);

You'll need to move const_check from fmt/format.h to fmt/core.h. In the future this if will become constexpr if.

vitaut · 2020-03-14T15:43:14Z

Maybe worth adding

dynamic_format_arg_store::reserve(size_t)

static_assert into void push_back(std::reference_wrapper<T> arg) to prevent passing primitive types by ref

But I'd leave those for the a PR with named args.

Yes, let's leave these for a separate PR.

`internal::std_string_veiew` alias

vsolontsov-ll · 2020-03-14T17:06:05Z

sorry, seems like I missed a few comments

Changed dispatching from tag to explicit `if`. However I would prefer tag-based dispatching if `if constexpr`, but the later is not available in older standards.

vsolontsov-ll · 2020-03-14T18:40:20Z

Ok, seems to be done. Sorry for that false-start.

vitaut

A few more comments (and there are several left over from previous review).

vitaut · 2020-03-15T21:31:46Z

include/fmt/core.h

+template <typename T> struct is_ref_wrapper : std::false_type {};
+
+template <typename T>
+struct is_ref_wrapper<std::reference_wrapper<T>> : std::true_type {};


Also let's rename this trait to is_reference_wrapper for consistency with the class name itself.

vitaut · 2020-03-15T21:35:38Z

include/fmt/core.h

+  // Workaround clang's -Wweak-vtables. For templates (unlike regular classes
+  // doesn't complain about inability to deduce translation unit to place vtable


I had problems parsing the above. How about rephrasing to something like:

// Workaround clang's -Wweak-vtables for templates (unlike regular classes) // don't complain about inability to deduce translation unit to place vtable.

Will fix.. Not exactly like you wrote but somehow... Hope it's better now

vitaut · 2020-03-15T21:38:43Z

include/fmt/core.h

+    Note that primitive type values are copied into basic_format_arg<>.
+    \endrst


I think we should just disallow passing reference wrappers of built-in types to prevent confusion.

vitaut · 2020-03-15T21:39:05Z

test/format-dyn-args-test.cc

+  // Unfortunately the tests are compiled with old ABI
+  // So strings use COW.


Why is this a problem?

Imagine a std::string with COW in shared state passed into dyn_arg_store by ref.
It takes fmt::string_view of the string. Next non-const subscription operator leads to COW, so string_view keeps pointing to an old version. So the test below won't work with old ABI std::string.

vitaut · 2020-03-16T14:00:38Z

Merged, thanks!

vsolontsov-ll added 2 commits March 9, 2020 20:50

Dynamic arguments storage. Implementation of enhancement from issue

7a1a67f

fmtlib#1170. First try...

Finxing build issues. (Not complete)

833bfe3

vsolontsov-ll closed this Mar 9, 2020

Fixed a lifetime issue in test and minor build fixes.

c3c6d0c

vsolontsov-ll reopened this Mar 10, 2020

vsolontsov-ll added 2 commits March 10, 2020 13:50

MSVC2015 fix

448fb74

Clang-format applied.

5930040

vitaut requested changes Mar 11, 2020

View reviewed changes

include/fmt/core.h Outdated Show resolved Hide resolved

include/fmt/core.h Outdated Show resolved Hide resolved

include/fmt/core.h Outdated Show resolved Hide resolved

include/fmt/dyn-args.h Outdated Show resolved Hide resolved

include/fmt/dyn-args.h Outdated Show resolved Hide resolved

vsolontsov-ll and others added 9 commits March 12, 2020 09:34

Merge branch 'master' of https://github.com/fmtlib/fmt.git

7433fd5

Addressed minor issues, removed named_arg support.

f0984c4

named_args will be added in a separate PR. Pending: avoid Args types from dynamic_format_arg_store template parameters, replace std::forward_list<std::variant> with a custom list.

Custom list -- first cut.

107f9a8

Pending: cleanup.

Update dyn-args.h

4351334

Removed constexpr

Build fixes

90219a7

Preliminary cleanup.

bee2b5f

Pending -- move dyn-args.h to core.h

Merge branch 'master' of https://github.com/fmtlib/fmt.git

c2c5c5c

minor (unused alias in test)

6e047c2

merge dyn-args.h to core.h

70bc839

vsolontsov-ll requested a review from vitaut March 14, 2020 10:02

vitaut requested changes Mar 14, 2020

View reviewed changes

vsolontsov-ll added 3 commits March 14, 2020 19:10

minor fixes

def562c

Removed conditional compilation as there's already the

49a46f8

`internal::std_string_veiew` alias

trying to fix MSVC

863aa63

vsolontsov-ll requested a review from vitaut March 14, 2020 17:05

Fixed comments

473ba57

Changed dispatching from tag to explicit `if`. However I would prefer tag-based dispatching if `if constexpr`, but the later is not available in older standards.

vitaut requested changes Mar 15, 2020

View reviewed changes

Fixes after review

ae512cd

vitaut merged commit 6012dc9 into fmtlib:master Mar 16, 2020

vitaut mentioned this pull request Mar 16, 2020

Add an API for dynamic construction of argument lists #1170

Closed

suo mentioned this pull request Apr 30, 2020

Dynamic construction of *named* argument lists? #1655

Closed

		template <typename T, typename Context>
		using need_dyn_copy_t = typename need_dyn_copy<T, Context>::type;

		using string_type = std::basic_string<char_type>;
		using value_type = basic_format_arg<Context>;

		// Storage of basic_format_arg must be contiguous
		// Required by basic_format_args::args_ which is just a pointer

		// Workaround clang's -Wweak-vtables. For templates (unlike regular classes
		// doesn't complain about inability to deduce translation unit to place vtable

		Note that primitive type values are copied into basic_format_arg<>.
		\endrst

		// Unfortunately the tests are compiled with old ABI
		// So strings use COW.

Dynamic arguments storage. Implementation of enhancement from issue #1170. #1584

Dynamic arguments storage. Implementation of enhancement from issue #1170. #1584

Conversation

vsolontsov-ll commented Mar 9, 2020

vsolontsov-ll commented Mar 10, 2020

vitaut left a comment

Choose a reason for hiding this comment

vsolontsov-ll commented Mar 14, 2020

vitaut left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vitaut commented Mar 14, 2020

vsolontsov-ll commented Mar 14, 2020

vsolontsov-ll commented Mar 14, 2020

vitaut left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vsolontsov-ll Mar 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vitaut commented Mar 16, 2020

vsolontsov-ll Mar 16, 2020 •

edited

Loading