unicode characters wrong representation

[matsievskiysv]

Here is a test script for gnuplot:

reset
set encoding utf8
set terminal pngcairo enhanced
set grid
set xlabel 'α'
set ylabel 'Δβ'
set output "./test.png"
plot [x=0:5] exp(-x)*cos(5*x) with lines title "I_{b} = 0,2 Ампер"

When I run this from command line with gnuplot test.gpi, I get the following output:

But when I open the same file in emacs and issue the command gnuplot-send-buffer-to-gnuplot, I get:

[Harry79]

I am invoking gnuplot from org-mode. This works fine:

#+begin_src gnuplot :file gnuplot-mode-issue-39.png
reset
set encoding utf8
set terminal pngcairo enhanced
set grid
set xlabel 'α'
set ylabel 'Δβ'
plot [x=0:5] exp(-x)cos(5x) with lines title "I_{b} = 0,2 Ампер"
#+end_src

#+RESULTS:
[[file:gnuplot-mode-issue-39.png]]

[matsievskiysv]

I use GNU Emacs 26.1, gnuplot 5.2 patchlevel 6. And this is my init script for gnuplot-mode. Maybe I messed up some configurations.

[Harry79]

Same Emacs, but Gnuplot Version 4.6 patchlevel 7.

[Harry79]

... on Windows by the way.

[conao3]

Windows is not supported for now. Ref: #33.

[matsievskiysv]

I don't use Windows

[matsievskiysv]

Problem is still there.
Linux 5.4.0-4-amd64 #1 SMP Debian 5.4.19-1 (2020-02-13) x86_64 GNU/Linux
GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.11) of 2019-09-22, modified by Debian

[mtreca]

Hmm. I also get a correct output with emacs -Q and the following configuration:

(require 'use-package)
(use-package gnuplot
  :load-path "~/.emacs.d/lib/gnuplot-20200317.131")

Can you try with emacs -Q as well?

[matsievskiysv]

Nope, I get the same output with emacs -Q

[rolandog]

I'm having the same problem (finally found this issue after a lot of searching!).

I'm using:

• Emacs: GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.16.0)
• Gnuplot: Version 5.4 patchlevel 1
• gnuplot-mode: 20220102.1637

I have tried so far:

• Using a magic comment of # -*- coding: utf-8; mode: org; -*-
• Setting in my init.el files:

  ;; add utf-8 at the front of the priority list for automatic detection
  (prefer-coding-system 'utf-8)
  
  ;; coding system to be used for encoding the buffer contents on saving
  (setq buffer-file-coding-system 'utf-8)
  
  ;; decide a coding system to use for a file I/O operation
  (add-to-list 'file-coding-system-alist '("\\.org\\'" . utf-8) t)
  
  ;; decide a coding system to use for a process I/O operation
  (add-to-list 'process-coding-system-alist '("gnuplot\\'" . utf-8) t)
  

Although I'm unsure if I entered process-coding-system-alist properly.

My steps to reproduce are:

1. emacs -Q.
2. run the following on *scratch*, or save to init-gnuplot-mode-issue-39.el:

;; -*- coding: utf-8; mode: elisp; -*-

;; current-language-environment is "English"
(set-language-environment "UTF-8")

;; sets the value of various coding systems
;; - coding system of a newly created buffer
;; - default coding system for subprocess I/O
;; - file-name-coding-system (if ASCII-compatible)
;; - set-terminal-coding-system
;; - set-keyboard-coding-system (if ASCII-compatible)
(set-default-coding-systems 'utf-8)

;; set clipboard and selection coding systems
(set-clipboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)

;; add utf-8 at the front of the priority list for automatic detection
(prefer-coding-system 'utf-8)

;; coding system to be used for encoding the buffer contents on saving
(setq buffer-file-coding-system 'utf-8)

;; decide a coding system to use for a file I/O operation
(add-to-list 'file-coding-system-alist '("\\.org\\'" . utf-8) t)

;; decide a coding system to use for a process I/O operation
(add-to-list 'process-coding-system-alist '("gnuplot\\'" . utf-8) t)

;; add all descendant directories of a directory to your load-path
;; https://www.emacswiki.org/emacs/LoadPath
(let ((default-directory  "~/.config/emacs/lisp/"))
  (normal-top-level-add-subdirs-to-load-path))

;; add path to elpa packages for purposes of esting with emacs -Q
(add-to-list 'load-path (directory-file-name "~/.config/emacs/elpa/gnuplot-20220102.1637"))

;; disable completions for gnuplot; otherwise we get:
;; run-hooks: Symbol's function definition is void: gnuplot-context-sensitive-mode
(setq gnuplot-use-context-sensitive-completion nil)

(autoload 'gnuplot-mode "gnuplot" "Gnuplot major mode" t)
(autoload 'gnuplot-make-buffer "gnuplot" "open a buffer in gnuplot-mode" t)
(add-to-list 'auto-mode-alist '("\\.gp\\'" . gnuplot-mode) t)

;; enable org-mode
(require 'org)

;; enable gnuplot-mode
(require 'gnuplot)

;; languages to make available in org-mode
(org-babel-do-load-languages
 'org-babel-load-languages
 '((gnuplot . t)))

3. Launch either emacs -Q gnuplot-mode-issue-39.org (and run the previous stuff on scratch), or emacs -Q --load gnuplot-mode-issue-39.el gnuplot-mode-issue-39.org
4. C-c C-c on the following gnuplot src block in gnuplot-mode-issue-39.org:

# -*- coding: utf-8; mode: org; -*-

#+begin_src gnuplot :file gnuplot-mode-issue-39.png
reset
set encoding utf8 # or set encoding locale
set terminal pngcairo enhanced
set grid
set xlabel "α"
set ylabel "Δβ"
plot [x=0:5] exp(-x)*cos(5*x) with lines title "I_{b} = 0,2 Ампер"
#+end_src

Note: I had to modify the equation a bit so that I didn't get some warnings, and I also had to disable gnuplot-use-context-sensitive-completion because it was also throwing some errors.

Edit: forgot to add gnuplot-mode version.

Edit 2: I forgot to mention that I tried both set encoding utf8 and set encoding locale with LC_ALL=en_US.UTF-8 emacs -Q --load gnuplot-mode-issue-39.el gnuplot-mode-issue-39.org to see if that would work (see Emacs docs, and the Gnuplot docs). But, I'm still getting the same image as the original commenter.

[rolandog]

By the way, if I switch to the *gnuplot* buffer and hit C-x RET f, I get:

Coding system for saving file (default iso-2022-jp):

I also forgot to add an image in the previous post:

Slightly different, but same artifacts.

Edit: decided to add the extra code that I mentioned here into the custom 'init.el' file above.

[rolandog]

In my case, what I could find is that---apparently---the words for the labels (the only text in my own file that had non-ascii characters) were being decoded in ISO-8859-1 (Latin-1) or possibly ISO-8858-15:

• The word Mínimo showed up as Mínimo
• The word Máximo showed up as Máximo

So, this is text encoded as UTF-8 that is somehow being decoded as ISO-8859-1 inside a UTF-8 encoded document (the svg).

If I switch to set encoding iso_8859_1 I somehow get an even 'worse' encoding:

• The word Mínimo showed up as MÃ�­nimo
• The word Máximo showed up as MÃ�¡ximo

I'm not sure if I'm wrapping my head around this properly, but I think that that would be the previous 'Mínimo' taken as UTF-8 and re-decoded as ISO-8859-1 inside an already ISO-8859-1 encoded document (the svg).

[mtreca]

Thank you for the detailed report @rolandog. I will try to reproduce your issue and investigate it, but I doubt I will be able to recreate it. I would wager that the encoding issue comes from the elisp side of things after calling `gnuplot-send-buffer-to-gnuplotˋ. I will look into it today, but I am not too familiar with the internals of the package.