[book] 500 internal server error for ja translation

[harupong]

Japanese translation https://git-scm.com/book/ja/v2 is returning 500 error. v1 translation https://git-scm.com/book/ja/v1 seems to be OK, so do all the other translations.

The oldest tweet mentioning this issue is posted at May 5 8:47AM UTC.

Since the translated source https://github.com/progit/progit2-ja has not been updated since Feb. 2016, I suspect that the recent changes added to git-scm.com are causing the issue.

[peff]

Thanks for reporting. It's fixed, at least for now. The problem is that importing the book seems to bring in a bogus "chapter 11" that has a single section with a nil title (and so enumerating the table of contents throws an exception).

The title of the bogus chapter is 索引. I don't know if that means anything to somebody who actually understands Japanese.

The ActiveRecord timestamp is from only a few days ago. @jnavila, could this be related to your #756?

I must admit I am puzzled why the ja version got rebuilt at all. We run rake genbook2 every 10 minutes, but AFAIK it should do nothing unless the source material was updated, or if GENLANG=ja is specified in the environment (at least that has been my experience, and I was force-rebuilding only the en version).

[peff]

So the reason I say "for now" is that I fixed it by manually deleting the bogus section from the database. But it may come back if whatever caused the regen a few days ago repeats itself. It looks like the trigger is actually the unsetting of the processed field on the book, which can happen only via somebody hitting the update HTTP endpoint with the correct auth token set. I have no idea if there are any automated processes which do that.

[jnavila]

I don't think my change would introduce a new chapter. Only new paragraphs are added (and only paragraphs). Nevertheless, I'll check with the hml files of ja.

[peff]

I wondered if maybe it sucked up a paragraph which used to become the first section, or something like that. I admit that's a wild guess, though.

[m-takagi]

索引 means Index in Japanese. It exists in book/index.asc.

[jnavila]

The index.html in the file https://progit2.s3.amazonaws.com/ja/2016-02-11-8cb2f/progit-ja.1016.zip has no content under the title. So maybe my patch introduces the inclusion of this file which did not happen before.

Edit: Does not seem so to me, the processing of the chapter is validated above the incriminated change.

[harupong]

Thanks for looking into it, you all. And thanks for the "for now" fix, @peff, the site is back now 😄

As @m-takagi said, the word 索引 appears in book/index.asc and there only. The ja version of this file is modified, unlike all the other translations(except italian translation). And I guess that might be causing:

a. generated PDF having chapter 11 but missing the actual content of "Index" section(same applies to it translation, whose book/index.asc file is also modified)
b. causing an exception in the site generation process(which does not seem to happen on it translation)

I can happily revert the modification on book/index.asc, or would you like to look further into the build process?

[harupong]

For your comparison, I'm posting images taken from the PDFs of it/ja translation.

it translated PDF having chapter 11

ja translated PDF having chapter 11

[jnavila]

@harupong

TLDR; I think you have to revert.

I checked the generated build.json in en and ja of the html files from atlas.. In the en version, the index.html file is tagged as index and as such not included into the processing, whereas in ja, it is tagged as a chapter and processed. So I think that the == Index entry fires up a special processing and if you translate it, it is no longer recognized.

There is a remaining point, it seems that Atlas is stuck again. There hasn't been any new build for two months, despite new commits in some translation repositories. That means that even if you revert, we cannot be sure that it was the primary cause of the failure.

@peff @schacon @ben What's the status of Atlas? Does it make sense to propose a patch for building the book in all languages directly from the github repos?

[peff]

@jnavila That makes sense.

I have no clue on the status of Atlas, or future plans for building the book.

[harupong]

@jnavila Thanks for the explanation. I've reverted the change, but as you mentioned, the build process doesn't seem to kick off yet.

[harupong]

@schacon @ben bump. Could you spare a minute to take a look at this issue?

Specifically, I'd love to hear your thoughts on @jnavila 's proposal to build the book in all languages directly from the github repos 👀

[jnavila]

Well, after some thoughts, it's not only building the html pages for the website, it's also providing the ebook and pdf versions, which involves a lot more dependencies and access rights (where the files are written).

[ben]

Let's discuss over at progit/progit2#625. Hopefully we'll be able to come up with a new toolchain and start publishing HTML again.

[pedrorijo91]

hey @jnavila, is this still a problem when importing the jap version of the book? or can we close this issue? nevertheless it seems more of a issue about book generation than the website itself?

[jnavila]

Not to my knowledge. Closing.