Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generate amalgamation file(with code not just idea) #863

Closed
gggin opened this issue Feb 23, 2017 · 8 comments
Closed

generate amalgamation file(with code not just idea) #863

gggin opened this issue Feb 23, 2017 · 8 comments

Comments

@gggin
Copy link

gggin commented Feb 23, 2017

I just want to an amalgamation file(like botan and sqlite or easyloggingpp).
Just an idea but with codes!
I hope that this idea can be merged into repo.

small question:

  • sqlite's homepage said that amalgamation's file will get a better performance but I don't know why!

Two file

  • merge.js
  • rapidjson-all.h

How to generate

  • firstput rapidjson-all.h and merge.js ==> here(in the picture)
    image
  • node merge.js
  • a file rapidjson-amalgamation.h be created

Usage

    #include "rapidjson-amalgamation.h"
    #include <iostream>

    int main()
    {
	rapidjson::Document doc;
	doc.Parse("{}");
	if (!doc.HasParseError())
	{
		rapidjson::StringBuffer buffer;
		rapidjson::Writer<rapidjson::StringBuffer> writer(buffer);
		doc.Accept(writer);
		auto a = buffer.GetString();
		std::cout << a << std::endl;
	}
	return 0;

Source code

rapidjson.zip

@StilesCrisis
Copy link
Contributor

I don't think this technique will be useful for a small template-based library. Have you actually measured a performance impact?

@gggin
Copy link
Author

gggin commented Mar 14, 2017

No, I have not measured.
This technique is easy for usage, just for easy usage.

@lichray
Copy link
Contributor

lichray commented Apr 11, 2017

Usually this technology is for projects with lots of translation units, to allow whole program optimization, but to header-only library, I don't think it matters.

About convenience, I always commit the whole rapidjson into my project, so I don't need it to be a single header.

@gggin
Copy link
Author

gggin commented Apr 12, 2017

Maybe nobody like this way to use rapidjson,so after a few time, I will close this comment.

@pinumbernumber
Copy link

I'm a fan of single-file amalgamations for libraries like this, although I would probably not use an unofficial one in case the process somehow introduced subtle issues.

@juj
Copy link

juj commented Sep 8, 2018

Thank you for this @gggin ! I was looking for an amalgamated version of RapidJSON for easy management, and this code was really useful. Noticed small change was needed to run merge.js on my Node.js 8.9.1: on line 12 with var con = content.toString(''); now needs to be var con = content.toString('utf-8');, due to nodejs/node#13936 .

sqlite's homepage said that amalgamation's file will get a better performance but I don't know why!

There are two conditions that come to mind when an amalgamation can have better (runtime) performance:

  1. If project consists of multiple compilation units and link-time optimizations (LTO) are not enabled in the compiler, then compiler will be limited in optimizing across compilation units and static library boundaries.
  2. Complex use of variables with static linkage in headers, producing duplicate definitions per compilation unit, that linker is unable to collapse.

Scenario 1 does not quite apply to all header projects, and if LTO is enabled in compiler (/LTCG in Visual Studio, --llvm-lto 1/2/3 in LLVM/Clang, flto in GCC), then it does not apply either. Scenario 2 requires somewhat difficult patterns, e.g. taking addresses of variables with static linkage, and passing them across function calls, that confuse optimizer - somewhat rare, doubt that occurs in many projects in practice.

It is probably the LTO bit why sqlite is referring to amalgamation having better performance. The actual benefit for having an amalgamation is more likely the convenience of integration into projects.

@gggin
Copy link
Author

gggin commented Sep 9, 2018

@juj Thank you for your response, If people need this, it maybe a good thing.

@im-infamou5
Copy link

Here is an amalgamated header right off the master.
Thx to @gggin and @juj
rapidjson-amalgamation.h.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants