Feature request: generate thebibliography environment

[ThomasA]

Biblatex is great, but unfortunately I sometimes submit papers to IEEE journals that require using BibTeX with IEEEtran.bst or pasting \bibitems into a thebibliography environment. The latter could be a shortcut to using Biblatex instead - if only I could make Biblatex generate \bibitems. Could this be possible?

[josephwright]

This is not really going to be possible in general I think. The way biblatex works is that data is read into LaTeX and that is then available for programmatic manipulation within the document. To create a file containing \bibitem entries you either need dedicated macros or to know that the manipulation is expandable. (Everything has to be able to be passed to a \write.) That's not the case in general and would be a significant change in a lot of the code. What one can do is bundle the data into the document and then continue to use biblatex for typesetting: http://tex.stackexchange.com/questions/12175/biblatex-submitting-to-a-journal. However, for journal submission even that is I think trisky: you should always use exactly what the journal asks for.

[odomanov]

@nickkolok suggested outputing right into pdf, for example using something like this:

\defbibenvironment{thebibliography}
  {\textbackslash begin\{thebibliography\}\{99\}
   \renewcommand*{\mkbibemph}[1]{\textbackslash textit\{##1\}}
   \list{}{}}
  {\endlist
   \textbackslash end\{thebibliography\}}
  {\item\textbackslash bibitem\{\printfield{entrykey}\}\\}

and then

\printbibliography[env=thebibliography]

This is already useful, you can just copy the text from the pdf file to the tex source. But maybe this can be adapted even futher?

An MWE to experiment:

\RequirePackage{filecontents}
\begin{filecontents*}{\jobname.bib}
@INBOOK{i1:aa,
 author = {Author},
 title = {Ztitle},
 booktitle = {Booktitle},
 editor = {Editor},
}
@BOOK{b1:ee-tt,
 author = {Eauthor},
 title = {Title},
}
@BOOK{b2_sss,
 author = {Sauthor},
 title = {Title},
}
\end{filecontents*}
\documentclass{article}
\usepackage[T1]{fontenc}
      
\usepackage{biblatex}
\addbibresource{\jobname.bib}

\defbibenvironment{thebibliography}
  {\textbackslash begin\{thebibliography\}\{99\}
   \renewcommand*{\mkbibemph}[1]{\textbackslash textit\{##1\}}
   \list{}{}}
  {\endlist
   \textbackslash end\{thebibliography\}}
  {\item\textbackslash bibitem\{\printfield{entrykey}\}\\}

\begin{document}
\nocite{*}
\printbibliography[env=thebibliography]
\printbibliography
\end{document}

[nickkolok]

Uploaded a package to CTAN: https://ctan.org/pkg/biblatex2bibitem
GitLab repo: https://gitlab.com/Nickkolok/biblatex2bibitem

[moewew]

That looks interesting. I take it this is an extension of the idea @odomanov presented above? The GitLab repo appears to be private (at least I get asked for a log-in). Can you share a short example that uses your package?

---

I decided to lock the discussion at #783 to bundle comments on one issue. In case people are interested in this issue they may still want to read the comments in #783 for some background.

[nickkolok]

I take it this is an extension of the idea @odomanov presented above?

Yes, surely. For the very initial history, refer to odomanov/biblatex-gost#20 (in Russian).

The GitLab repo appears to be private (at least I get asked for a log-in).

Sorry, it was my fault. Now it should be public.

[nickkolok]

Can you share a short example that uses your package?

Here you are: https://gitlab.com/Nickkolok/biblatex2bibitem/-/blob/master/mwe.tex

The PDF file looks like this:

[nickkolok]

Added more examples with formulas to the same file

[nickkolok]

In the example the output is useful enough, but in general there are a lot more commands that would need to be redefined (\mkbibquote, \mkbibbold, ...). And conceptually it is not clear where exactly (at what level of meaning "expansion") one would stop: Should we also output \newunitpunct instead of the result . ? What about \mkbibnamefamily?

The answer is really clear: the output should be compatible with the \bibitem format provided by \cite package. Thus, the output should not contain biblatex-only commands such as \mkbibnamefamily. If a command can be easily preserved (e.g. \mkbibemph -> \emph), itshould be preserved; otherwise it should be just executed. (If someone redefines e.g. \mkbibemph in his document, it is his own problem.)

But the more problematic issue (for inclusion in the core anyway) is that biblatex does not use thebibliography internally.

Well, looks as a missing fallback! Sometimes I'd prefer to format one reference manually (because of lack of time) and to get others formatted by biblatex.

Suppose you manage to export your bibliography into a thebibliography. You could not use that data as a drop-in replacement for anything (I assume one would hope to get rid of \addbibresource analogous to how you can just replace \bibliographystyle and \bibliography with thebibliography in BibTeX) since biblatex can not use the data from thebibliography.

It would be great if biblatex could cite bibitems as is. The use-case: one item is not formatted properly by biblatex, but the author has no time (or expertise) to fight with biblatex styles. Then the author just creates a thebibliography environment with a single item.

Anyway, it is a topic for a separate feature request.

In this scenario the only reason to use biblatex at all (as opposed to BibTeX) is to obtain a properly formatted thebibliography, but .bst files do that sort of work much better especially given the difficulties of finding the right sort of expansion level to write to the PDF. Now I understand that one might still think that the interface biblatex and Biber offer for these sort of things is simpler than the reverse Polish notation of .bst files. But I feel that if this feature were included in the biblatex core, it would have to be universally applicable and it would have to have the aspiration to work well for all sorts of styles and not just numeric styles.

My use-case is pretty simple: I know how to configure biblatex but don't know how to write .bst files. Another use-case is when I have found an appropriate biblatex style written by someone - and no bibtex style for the same. Then, if a journal accepts only \bibitem, I have to cast this hack.

Given that the only reason I can see for exporting thebibliography is journal submissions, where often there are ready-made templates that don't use biblatex anyway, I don't see a pressing need for this sort of thing.

Agree completely.

Summing up: your arguments against including the feature into the core of biblatex are more than sufficient :) However, you may still want to mention the biblatex2bibitem package in documentation or in FAQ.

[nickkolok]

There are some tricky issues that you may want to help with: https://gitlab.com/Nickkolok/biblatex2bibitem/issues

[moewew]

In the example the output is useful enough, but in general there are a lot more commands that would need to be redefined (\mkbibquote, \mkbibbold, ...). And conceptually it is not clear where exactly (at what level of meaning "expansion") one would stop: Should we also output \newunitpunct instead of the result . ? What about \mkbibnamefamily?

The answer is really clear: the output should be compatible with the \bibitem format provided by \cite package. [...]

I guess that is one sensible way to define what to do, yes. (Note that the cite package doesn't actually do anything to thebibliography or \bibitem, so this is the same as saying "commands defined in the LaTeX kernel and sensible document classes".) You could of course try to retain the biblatex wrappers and some commands (for example \newunitpunct) and load a small package that defines these commands. That way it would be possible to configure the thebibliography output after it has been produced. Depending on what you have in mind this may actually be quite useful.

But the more problematic issue (for inclusion in the core anyway) is that biblatex does not use thebibliography internally.

Well, looks as a missing fallback! Sometimes I'd prefer to format one reference manually (because of lack of time) and to get others formatted by biblatex.

[...]

It would be great if biblatex could cite bibitems as is. The use-case: one item is not formatted properly by biblatex, but the author has no time (or expertise) to fight with biblatex styles. Then the author just creates a thebibliography environment with a single item.

Anyway, it is a topic for a separate feature request.

As I said, biblatex absolutely does not use thebibliography or \bibitems internally, since thebibliography simply doesn't have enough structure for biblatex. thebibliography is essentially a glorified enumerate that (usually) holds printable text, \bibitem is a \label and \cite is a \ref. It is not possible to extract all bibliographic data that biblatex has available from a thebibliography in the normal format. You can read more about that in the first (background) part of my answer to https://tex.stackexchange.com/q/459161/35864 and between the lines also in https://tex.stackexchange.com/q/473471/35864. The upshot is that there is no way for biblatex to interact with classical thebibliography and \bibitems in a useful way.

The closest way to having most entries formatted by biblatex and adding a few manual references is via the methods described in https://tex.stackexchange.com/q/438759/35864 and #884.

---

There are some tricky issues that you may want to help with: https://gitlab.com/Nickkolok/biblatex2bibitem/issues

I'm afraid I won't have time to contribute actively to your project in the near future. But for https://gitlab.com/Nickkolok/biblatex2bibitem/issues/1 you may want to look at the field formats for DOIs and URLs in biblatex.def

biblatex/tex/latex/biblatex/biblatex.def

Lines 481 to 485 in 2ddd283

	\DeclareFieldFormat{doi}{%
	\mkbibacro{DOI}\addcolon\space
	\ifhyperref
	{\href{https://doi.org/#1}{\nolinkurl{#1}}}
	{\nolinkurl{#1}}}

biblatex/tex/latex/biblatex/biblatex.def

Line 568 in 2ddd283

\DeclareFieldFormat{url}{\mkbibacro{URL}\addcolon\space\url{#1}}

biblatex uses the commands \url (defined by url/hyperref) and \nolinkurl (defined by hyperref and set to \providecommand*{\nolinkurl}{\url} otherwise by biblatex). So you'd have to redefine those commands (which might be a bit tricky because you don't want to destroy their ability to accept special characters) or meddle with the field formats. In any case you'd need to make sure your document using thebibliography loads the url package or defines a fallback for \url.

\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{babel}
\usepackage{csquotes}

\usepackage[style=authoryear, backend=biber]{biblatex}

\addbibresource{biblatex-examples.bib}

\DeclareFieldFormat{url}{\mkbibacro{URL}\addcolon\space\ttfamily \textbackslash url\{\url{#1}\}}

\begin{document}
\cite{markey}
\printbibliography
\end{document}

[nickkolok]

Thank you very much for pointing out to \nolinkurl, \url and \href! It looks to be enough to define the commands as self-printing:

\renewcommand*{\nolinkurl}[1]{\textbackslash nolinkurl \{##1\}}%
\renewcommand*{\url}[1]{\textbackslash url \{##1\}}%
\renewcommand*{\href}[2]{\textbackslash href \{##1\} \{##2\}}%

The only problem is that the rows are too long and thus not copyable...
However, no one forbids to change page geometry before the copyable bibliography.
(\sloppy did not help for me).

[moewew]

If URLs are typeset as normal text, TeX will have a hard time figuring out where to break/hyphenate them. That's why in my solution I tried to still use \url in its original definition internally, that should allow for line breaks in the URL..
Maybe something like

\let\blxiibitm@nolinkurl\nolinkurl
\renewcommand*{\nolinkurl}[1]{\textbackslash nolinkurl\{\blxiibitm@nolinkurl{#1}\}}
\renewcommand*{\url}[1]{\textbackslash url\{\blxiibitm@nolinkurl{#1}\}}

can help.

[alex-ball]

Just for the record, and with no expectation of reviving this issue, I wanted to add another niche use case to this conversation.

As a style author looking to replicate a particular specification, I have a sets of tests consisting of a manually formatted reference and the automated output from LaTeX. Up till now, I have been comparing them by eye, and not always accurately. Having found a way to automate the comparison process with a CSL style I maintain, I have been looking at how I could to the same with LaTeX. With BibTeX it turned out to be relatively straightforward to post-process the .bbl file, but I struggled with how to get the biblatex output into a form I could use. Then just recently I stumbled on this thread and found that the output from biblatex2bibitem worked the best of the options I tried. It would be preferable to get the output as plain text to avoid artefacts from the PDF (such as blank lines from page breaks and truncated unbreakable strings) but these things can be worked around.

As I say, I have no expectation of any action, but I wanted to point out there is another use for this sort of output. If there are other ways of programmatically testing the output of biblatex I've missed (which is quite likely) I would be interested to hear about them.

[moewew]

If you have the typeset LaTeX "target" ready (e.g. because you know that the output currently produced by the style is correct), you can use \showoutput-based comparison with l3build. @josephwright has just implemented this for the biblatex core (#1207) and I have been using it for a while also for biblatex-ext (https://github.com/moewew/biblatex-ext/tree/dev/testfiles).

The downside to \showoutput is that the output is massive and it can be hard to spot the actual changes. On the plus side, the output is pretty detailed and shows you stuff that you might miss if you just compare visually or after text extraction from PDF.

[nickkolok]

It would be preferable to get the output as plain text to avoid artefacts from the PDF (such as blank lines from page breaks and truncated unbreakable strings) but these things can be worked around.

1. Put a pagebreak before the bibliography, if there isn't
2. Set the smallest possible font and the largest possible paper size
3. Cut off all the pages before the bibliography - e.g. with gs
4. Use smth like textractor to extract the text from PDF

[dcpurton]

If you have the typeset LaTeX "target" ready (e.g. because you know that the output currently produced by the style is correct), you can use \showoutput-based comparison with l3build. @josephwright has just implemented this for the biblatex core (#1207) and I have been using it for a while also for biblatex-ext (https://github.com/moewew/biblatex-ext/tree/dev/testfiles).

The downside to \showoutput is that the output is massive and it can be hard to spot the actual changes. On the plus side, the output is pretty detailed and shows you stuff that you might miss if you just compare visually or after text extraction from PDF.

For an example of this I've started using this for the next version of biblatex-sbl. See https://github.com/dcpurton/biblatex-sbl/tree/v2

[alex-ball]

I'm going to stick with the biblatex2bibitem route for now because (a) from my perspective getting the textual appearance right is the important thing, not so much the precise stretchability of the white space, etc. and (b) with the University of Bath style in particular it is quite a volatile target, and when it changes it is useful if I only have to change the target text output (which is in the documentation and also used for testing BibTeX and CSL) and not \showoutput output in addition.

I have working solutions set up such that a Python script compares the output against the target text in the documentation for me:

• biblatex-bath: see https://github.com/alex-ball/bathbib – /check.py, /biblatex/test-output.tex and /biblatex/Makefile (recipes for *.bbi files)
• biblatex-oxref: see https://github.com/alex-ball/biblatex-oxref – /check.py, /test-template.tex and /Makefile (recipe for *.bbi files)

But it is very helpful to have examples of the \showoutput-based testing method for more stringent analysis, so thank you for sharing.