Additional tufte ebook error

[RudolfLi]

After installing the rticles package I tried to make an example tufte ebook using the template. After installing the dependencies (note: those were not automatically installed upon loading the rticles package) I got the following error (a different one from the one reported by AmitRohanty)

pandoc-citeproc.exe: Cannot decode byte '\xf3': Data.Text.Internal.Encoding.Fusion.streamUtf8: Invalid UTF-8 stream
pandoc.exe: Error running filter pandoc-citeproc
Error: pandoc document conversion failed with error 83

The complete log is:

---

processing file: Mytitle.Rmd
|.. | 5%
ordinary text without R code

|.... | 11%
label: setup (with options)
List of 4
$ echo : logi FALSE
$ message: logi FALSE
$ cache : logi FALSE
$ include: logi TRUE

|...... | 16%
ordinary text without R code

|....... | 21%
label: unnamed-chunk-1 (with options)
List of 2
$ fig.margin: logi TRUE
$ fig.cap : chr "Sepal length vs. petal length, colored by species"

|......... | 26%
ordinary text without R code

|........... | 32%
label: unnamed-chunk-2 (with options)
List of 4
$ fig.width : num 10
$ fig.height : num 2
$ fig.fullwidth: logi TRUE
$ fig.cap : chr "Full width figure"

|............. | 37%
ordinary text without R code

|............... | 42%
label: unnamed-chunk-3 (with options)
List of 1
$ fig.cap: chr "Another figure"

|................. | 47%
ordinary text without R code

|.................. | 53%
label: unnamed-chunk-4 (with options)
List of 1
$ results: chr "asis"

|.................... | 58%
ordinary text without R code

|...................... | 63%
label: amargintable (with options)
List of 2
$ results: chr "latex"
$ echo : logi FALSE

|........................ | 68%
inline R code fragments

|.......................... | 74%
label: pandocDemo2 (with options)
List of 1
$ results: chr "asis"

|............................ | 79%
inline R code fragments

|............................. | 84%
label: kable (with options)
List of 1
$ results: chr "asis"

|............................... | 89%
inline R code fragments

|................................. | 95%
label: fullwidthxtable (with options)
List of 2
$ results: chr "asis"
$ echo : logi FALSE

|...................................| 100%
inline R code fragments

"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS Mytitle.utf8.md --to latex --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash-implicit_figures --output Mytitle.tex --filter pandoc-citeproc --table-of-contents --toc-depth 3 --template "D:\bin\R\R-3.2.2\library\rticles\rmarkdown\templates\tufte_ebook\resources\tufte-ebook.tex" --number-sections --highlight-style pygments --latex-engine xelatex --bibliography knitcitations.bib
Writing 8 Bibtex entries ... OK
Results written to file 'knitcitations.bib'
output file: Mytitle.knit.md

pandoc-citeproc.exe: Cannot decode byte '\xf3': Data.Text.Internal.Encoding.Fusion.streamUtf8: Invalid UTF-8 stream
pandoc.exe: Error running filter pandoc-citeproc
Error: pandoc document conversion failed with error 83
In addition: Warning message:
running command '"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS Mytitle.utf8.md --to latex --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash-implicit_figures --output Mytitle.tex --filter pandoc-citeproc --table-of-contents --toc-depth 3 --template "D:\bin\R\R-3.2.2\library\rticles\rmarkdown\templates\tufte_ebook\resources\tufte-ebook.tex" --number-sections --highlight-style pygments --latex-engine xelatex --bibliography knitcitations.bib' had status 83

Execution halted

[RudolfLi]

In itself R, Rstudio, the packages knitr, pandoc, rticles and the Latex compiler work properly as I was able to generate a document based on the JSS template (also included with rticles) without any problems.

Let me know if any additional information is needed.

[yihui]

It appears you used the R package knitcitations. You need to make sure it writes out the bib entries in UTF-8 (contact the package author if you don't know what this means).

[RudolfLi]

@yihui
Thanks for your reaction.

Err ... I don't feel I was 'using' the knitcitations package. My point is: it's loaded in the tufte book example, and the example I ran (as supplied) doesn't work. Commenting knitcitations out doesn't work because of all the function calls (citep) the document contains. Removing the citep calls doesn't work either: I get a downstream complaint that the bib file must be present.

Looking at the document I see that knitcitations does something with citations, but I didn't see what exactly (I found the bib database that's submitted to Latex, but seems to be generated by pandoc, not knitcitations. I couldn't find out where pandoc gets its entries from (they're not in the example Rcmd file) and I'm not sure how all this ties in with knitcitations either.). A quick online search on the problem suggests that somehow the knitcitations finds a bibliographic reference with 'funny' characters, puts those characters in the bibliography and causes downstream packages to choke. The package maintainer of knitcitations responded to an earlier question (april 2015) about non utf-8 chars by blaming the 'choice of 'locale' in Windows and said he couldn't reproduce the problem.

Trying to edit the bibtex database to remove the funny characters doesn't work, as it's overwritten each time you 'knit' the example. I have no idea what citation is the problem as none of the program logs is telling me that.

I just know the basics about Latex and barely anything about pandoc. I still think it's something for the rticles package maintainer to resolve. I ran the example in order to learn how to use rticles, not to do substantial debugging on one of its examples due to package incompatibilities when I know barely anything about the toolchain involved. Does that seem reasonable?

[yihui]

Unfortunately we do not maintain the knitcitations package, so we cannot solve the problem on our side. If you claim you were not using knitcitations package, please prove it by providing a minimal reproducible example. If the problem still occurs when no third-party packages are used, that means it is rticles' responsibility to fix the problem, otherwise it will be too much burden for us to fix the whole outside world...

[RudolfLi]

@yihui
Quite right :-).

(1) Just load the tufte e-book example as provided by the rticles package, without any alterations or additions, and knit it. That should reproduce the error.

(2) As I wrote, I didn't 'use' the knitcitations package, the tufte ebook example provided by rticles does, right out of the box.

What I can do on my side is to to try and re-introduce the citations from the rticles example (I removed all of them) and see if I can find one that doesn't cause a problem. Does that sound like a plan?

[RudolfLi]

@yihui

Hi Yihu, got it.
It's the reference r citep(citation("pander")) in your example file that gums up the works.
The author's name contains an 'o' with an accent (I caught onto that one by examining the bibtex file when the only reference I had was to pander), and that's something some program somewhere in the chain can't handle.

I left the following references in place:
r citep(citation("xtable")); r citep(citation("knitr"))
r citep(citation("knitcitations")); r citep(citation("RefManageR"))
and all is well.

I still have no idea where the rcites package got that name from. A hint would be welcome.

[jjallaire]

@justsomeone1001
https://github.com/rstudio/rticles/commits/master/inst/rmarkdown/templates/tufte_ebook/skeleton?author=justsomeone1001
Could
you help out here?

On Sat, Nov 14, 2015 at 11:25 AM, RudolfLi notifications@github.com wrote:

@yihui https://github.com/yihui
Thanks for your reaction.

Err ... I don't feel I was 'using' the knitcitations package. My point is:
it's loaded in the tufte book example, and the example I ran (as supplied)
doesn't work. Commenting knitcitations out doesn't work because of all the
function calls (citep) the document contains. Removing the citep calls
doesn't work either: I get a downstream complaint that the bib file must be
present.

Looking at the document I see that knitcitations does something with
citations, but I didn't see what exactly (I found the bib database that's
submitted to Latex, but seems to be generated by pandoc, not knitcitations.
I couldn't find out where pandoc gets its entries from (they're not in the
example Rcmd file) and I'm not sure how all this ties in with knitcitations
either.). A quick online search on the problem suggests that somehow the
knitcitations finds a bibliographic reference with 'funny' characters, puts
those characters in the bibliography and causes downstream packages to
choke. The package maintainer of knitcitations responded to an earlier
question (april 2015) about non utf-8 chars by blaming the 'choice of
'locale' in Windows and said he couldn't reproduce the problem.

Trying to edit the bibtex database to remove the funny characters doesn't
work, as it's overwritten each time you 'knit' the example. I have no idea
what citation is the problem as none of the program logs is telling me that.

I just know the basics about Latex and barely anything about pandoc. I
still think it's something for the rticles package maintainer to resolve. I
ran the example in order to learn how to use rticles, not to do substantial
debugging on one of its examples due to package incompatibilities when I
know barely anything about the toolchain involved. Does that seem
reasonable?

—
Reply to this email directly or view it on GitHub
#51 (comment).

[elbamos]

Yeah, I'll take care of this this weekend.

[elbamos]

Ok, I just looked at this. It compiles perfectly here.

However, there have been other documents where I have seen the "error 83" caused by special characters in a bibtex. I have not seen them in about a year though -- I believed it was a bug in part of the TeX distribution that had been fixed.

I suspect this is going to turn out to be a limitation of the Windows implementation of pandoc or some part of TeX.

@RudolfLi Can you provide more information about your TeX & pandoc installation? Versions? If you have a way of reaching you directly, I'd like to try to track this down once and for all.

[yihui]

I feel this is more likely to be an issue of knitcitations instead of Pandoc. knitcitations has to write the output in UTF8. If it is too much trouble to get it right, I'd suggest we remove knitcitations from the example document. It is not essential after all.

[elbamos]

I see your point, but I’ve seen this error before.  Let me try to track it down with him.  You may be right. If its being caused by knitcitations then of course knitcitations gets pulled out. 

From: Yihui Xie notifications@github.com
Reply: rstudio/rticles reply@reply.github.com
Date: November 14, 2015 at 6:19:59 PM
To: rstudio/rticles rticles@noreply.github.com
CC: elbamos amos.elberg@me.com
Subject:  Re: [rticles] Additional tufte ebook error (#51)

I feel this is more likely to be an issue of knitcitations instead of Pandoc. knitcitations has to write the output in UTF8. If it is too much trouble to get it right, I'd suggest we remove knitcitations from the example document. It is not essential after all.

—
Reply to this email directly or view it on GitHub.

[RudolfLi]

@elbamos

Latex: Miktex 2.9
Pandoc: I installed the "pander" package from github earlier today:

---

Package: pander
Maintainer: Gergely Daróczi daroczig@rapporter.net
Title: An R Pandoc Writer
Type: Package
Encoding: UTF-8
Description: Contains some functions catching all messages, stdout and other
useful information while evaluating R code and other helpers to return user
specified text elements (like: header, paragraph, table, image, lists etc.)
in pandoc's markdown or several type of R objects similarly automatically
transformed to markdown format. Also capable of exporting/converting (the
resulting) complex pandoc documents to e.g. HTML, PDF, docx or odt. This
latter reporting feature is supported in brew syntax or with a custom
reference class with a smarty caching backend.
Author: Gergely Daróczi daroczig@rapporter.net, Roman Tsegelskyi
roman.tsegelskyi@gmail.com
Version: 0.5.2
Date: 2015-04-19
URL: http://rapporter.github.io/pander
BugReports: https://github.com/rapporter/pander/issues
License: AGPL-3 | file LICENSE
Depends: R (>= 2.15.0)
Imports: methods, digest, tools, Rcpp
Suggests: grid, lattice, ggplot2 (>= 0.9.2), koRpus, survival,
microbenchmark, zoo, nlme, descr, MASS, knitr, tables
SystemRequirements: pandoc (http://johnmacfarlane.net/pandoc) for
exporting markdown files to other formats.
LinkingTo: Rcpp
Packaged: 2015-05-16 21:27:58 UTC; daroczig
NeedsCompilation: yes
Repository: CRAN
Date/Publication: 2015-05-18 07:34:49
Built: R 3.2.2; x86_64-w64-mingw32; 2015-11-13 04:16:53 UTC; windows

Archs: i386, x64

Let me know if there is any more information I can help you with.

P.S. as I mentioned, the issue might have something to do with locales under Windows (whatever they are), as voiced by the author of the knitcitations package here:

cboettig/knitcitations#74

P.P.S. If you omit the special characters, it works OK. If you compile your bibtex database manually, you can ensure that no such characters are present. The problem is that knitcitations seems to have license to obtain a bibtech reference from ... whereever. So you have no control over special characters or anything.

P.P.P.S. Ironic isn't it, that it's the name of pander's author (that accented 'o') that causes pandoc to choke :-).

[elbamos]

@RudolfLi well done, yeah I'm sure that's exactly what it is. I've asked on that issue to see what the status is and we can proceed from there.

You are using R 3.2, right?

I'm not a fan of pulling something out because of an issue with supporting Windows. There are so many different Windows flavors, and Windows support for R and TeX is just not good.

If there hasn't been a resolution to this, I'll put some code into the template to detect the OS and report the issue while skipping that citation.

[RudolfLi]

@elbamos

My R version is:

---

sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices datasets utils methods base

other attached packages:
[1] knitr_1.11 markdown_0.7.7 installr_0.17.0

loaded via a namespace (and not attached):
[1] httr_1.0.0 R6_2.1.1 magrittr_1.5 htmltools_0.2.6 tools_3.2.2 rticles_1.0
[7] curl_0.9.3 yaml_2.1.13 memoise_0.2.1 stringi_1.0-1 rmarkdown_0.8.1 stringr_1.0.0
[13] digest_0.6.8 devtools_1.9.1

---

[csrvermaak]

👍

[elbamos]

Ok. Knitcitations isn't going to fix it, and since there apparently is more than one Windows user out there still (seriously, people...), I'm going to put in code to turn off that cite and issue a warning on Windows.

[csrvermaak]

Thanks @elbamos, much appreciated

[jcpsantiago]

I'm using Arch Linux x64 and I get this error:

pandoc: Error running filter /usr/lib/rstudio/bin/pandoc/pandoc-citeproc
fd:6: hClose: resource vanished (Broken pipe)
Error: pandoc document conversion failed with error 83
Execution halted

I simply installed the rticles pakcage in RStudio, along with dependencies and tried to knit it

removing the line: bibliography: "knitcitations.bib" allows the document to be compiled. I'm actually trying to use the tufte handout template outside of R, because I just want to write a protocol in markdown with svg images, but that seems to be way more complex than simply starting a Latex document

[elbamos]

@elgitmariachi Assuming that its tufte-ebook you're trying to knit, can you open up a new issue, and try upgrading your installation of LaTeX and pandoc to the latest through Arch's package management system? Thanks.

[yihui]

Closing this issue since the tufte-ebook format has been removed from this package (b5a41d0). You may consider the tufte package instead: https://github.com/rstudio/tufte.

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.