#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# latex2unicode.py: Convert a simple inline TeX/LaTeX (aimed at ArXiv abstracts) into Unicode+HTML+CSS, using the OA API.
# Author: Gwern Branwen
# Date: 2023-06-28
# When: Time-stamp: "2023-11-13 17:49:09 gwern"
# License: CC-0
#
# Usage: $ OPENAI_API_KEY="sk-XXX" xclip -o | python latex2unicode.py
#
# Typesetting TeX/LaTeX for web browsers is typically a heavyweight operation; even if done server-side, display often requires a lot of CSS+fonts. And then the result looks highly unnatural and clearly 'alien', interrupting reading flow. This is worthwhile for complex equations, where browser typesetting is not up to snuff, but for many in-the-wild TeX uses, the use is often as simple as `$X$`, which would look better as `*X*` & take megabytes less to render. So it is desirable for simple TeX expressions to convert them to 'native' Unicode/HTML (augmented with a bit of custom CSS to handle things like superscripts-over-subscripts which pop up in integrals/summations/binomials/matrices etc).
# Unfortunately, TeX is an irregular macro language which is hard to parse and 'compile' to Unicode: it's easy to do many examples, but there's a long tail of weird variables, formatting commands etc, which means that I wind up defining lots of rewrites by hand, even though they are usually pretty 'obvious'. So, quite tedious and unrewarding.
# However, this is a perfect use-case for GPT models: it is hard to write comprehensive rules for, but is an extremely constrained problem in a domain it knows well which requires processing few tokens, where I can give it many few-shot examples, interrogate it for edge-cases to then write rules/examples for, and the harm of an error is relatively minimal (anyone seriously using an equation will need to read the original anyway, so won't be fooled by a wrong translation).
# So we write down a list of general rules, then a bunch of specific examples, then ask GPT-4 to translate from TeX to Unicode/HTML/CSS.
#
# eg.
# $ echo 'a + b = c^2' | python3 latex2unicode.py
# *a* + *b* = *c*^{2}
import sys
from openai import OpenAI
client = OpenAI()
if len(sys.argv) == 1:
target = sys.stdin.read().strip()
else:
target = sys.argv[1]
prompt = """
Task: Convert LaTeX inline expressions from ArXiv-style TeX math to inline Unicode+HTML+CSS, for easier reading in web browsers.
Details:
- Convert only if the result is unambiguous.
- Note that inputs may be very short, because each LaTeX fragment in an ArXiv abstract is processed individually. Many inputs will be a short as single letters (which are variables).
- Assume only default environment settings with no redefinitions or uses like `\newcommand` or `\begin`.
- Do not modify block-level equations, or complex structures such as diagrams or tables or arrays or matrices (eg `\begin{bmatrix}`), or illustrations such as drawn by TikZ or `\draw` , as those require special processing. Do not convert them & simply repeat it if the input is not an inline math expression.
- If a TeX command has no reasonable Unicode equivalent, such as the `\overrightarrow{AB}`/`\vec{AB}` or `\check{a}` or `\\underline`/`overline` commands in LaTeX, simply repeat it.
- If a TeX command merely adjusts positioning, size, or margin (such as `\big`/`\raisebox`/`\big`/`\Big`), always omit it from the conversion (as it is probably unnecessary & would need to be handled specially if it was).
- The TeX/LaTeX special glyphs (`\TeX` & `\LaTeX`) are handled elsewhere; do not convert them, but simply repeat it.
- Use Unicode entities, eg. MATHEMATICAL CAPITAL SCRIPT O πͺ in place of `\mathcal{O}`, and likewise for the Fraktur ones (`\mathfrak`) and bold ones (`\mathbb`). Convert to the closest Unicode entity that exists. Convert symbols, special symbols, mathematical operators, and Greek letters. Convert even if the Unicode is rare (such as `πͺ`). If there is no Unicode equivalent (such as because there is not a matching letter in that font family, or no appropriate combining character), then do not convert it.
- If there are multiple reasonable choices, such as `\approx` which could be represented as `β` or `~`, choose the simpler-looking one. Do not choose the complex one unless there is some good specific reason for that.
- For superimposed subscript+superscript, use a predefined CSS class `subsup`, eg. `(\Delta^0_n)` β `Ξ^{0}_{n}`; `\Xi_{cc}^{++} = ccu` β `Ξ_{cc}^{++} = *ccu*`; `\,\Lambda_c \Lambda_c \to \Xi_{cc}^{++}\,n\,` β `*Ξ*_{c} *Ξ*_{c} β Ξ_{cc}^{++},*n*`. This is also useful for summations or integrals, such as `\int_a^b f(x) dx` β `β«_{a}^{b} *f*(*x*) *dx*`.
- For small fractions, use FRACTION SLASH (β) to convert (eg. `1/2` or `\frac{1}{2}` β `1β2`). Do not use the Unicode fractions like VULGAR FRACTION ONE HALF `Β½`.
- For complex fractions which use superscripts or subscripts, multiple arguments etc, do not convert them & simply repeat them. eg. do not convert `\(\frac{a^{b}}{c^{d}}\)`, as it is too complex.
- Convert roots such as square or cube roots if that would be unambiguous. For example, `\sqrt[3]{8}` β `β8` is good, but not `\sqrt[3]{ab}` because `β*ab*` is ambiguous; do not convert complex roots like `\sqrt[3]{ab}`.
- Color & styling: if necessary, you may use very simple CSS inline with a `` declaration, such as to color something blue using ``.
- Be careful about dash use: correctly use MINUS SIGN (β) vs EM DASH (β) vs EN DASH (β) vs hyphen (-).
- More examples: ` O(1)` β ` πͺ(1)`; `\(\mathsf{TC}^0\)` β `**TC**^{0}`; `\(\approx\)` β `~`; `\(1-\tilde \Omega(n^{-1/3})\)` β `1 β Ξ©Μ(*n*^{β1β3})`; `\(\mathbf{R}^3\)` β `π^{3}`; `\(\ell_p\)` β `π_{p}`; `\textcircled{r}` β `β‘`; `(\nabla \log p_t\)` β `β log *p*_{t}`; `\(\partial_t u = \Delta u + \tilde B(u,u)\)` β `β_{t}*u* = Ξ*u* + *BΜ*(*u*, *u*)`; `\(1 - \frac{1}{e}\)` β `"1 β 1β*e*`; `O(\sqrt{T}` β `πͺ(β*T*)`; `\(^\circ\)` β `Β°`; `\(^\bullet\)` β `β’`; `6\times 10^{-6}\)` β `6Γ10^{β6}`; `5\div10` β `5 Γ· 10`; `\Pr(\text{text} | \alpha)` β `Pr(text | Ξ±)`; `\(\hbar\)` β `β`; `\frac{1}{2}`β `1β2`; `\nabla` β `β`; `\(r \to\infty\)` β `*r* β β`; `\hat{a}` β `Γ’`; \textit{zero-shot}` β `*zero-shot*`; `\(f(x) = x \cdot \text{sigmoid}(\beta x)\)` β `*f(x)* = *x* Γ sigmoid(Ξ² *x*)`; `\clubsuit` β `β£`; `\textcolor{red}{x}` β `x`; `\textbf{bolding}` β `**bolding**`; `\textit{emphasis}` β `*emphasis*`; `B` β `*B*`; `u` β `*u*`; `X + Y` β `*X* + *Y*`; `\,\Lambda_b \Lambda_b \to \Xi_{bb}\,N\,` β `, *Ξ*_{b} *Ξ*_{b} β Ξ_{bb} *N*,`, `x \in (-\infty, \infty)` β `x β (-β, β)`, `p\bar{p} \to \mu^+\mu^-` β `ppΜ
β ΞΌ^{+}ΞΌ^{β}`, `\alpha\omega\epsilon\S\om\in` β `Ξ±ΟΞ΅Β§ΓΈmβ`, `^2H ^6Li ^{10}B ^{14}N` β `^{2}H ^{6}Li ^{10}B ^{14}N`, `\mathcal{L} \mathcal{H} \mathbb{R} \mathbb{C}` β `β β β β`.
Task example:
Input to convert: \(H\gg1\)
Converted output: *H* β« 1
Task:
Input to convert: """ + target + "Converted output: "
completion = client.chat.completions.create(
model="gpt-4-1106-preview", # we use GPT-4 because the outputs are short, we want the highest accuracy possible, we provide a lot of examples & instructions which may overload dumber models, and reviewing for correctness can be difficult, so we are willing to spend a few pennies to avoid the risk of a lower model
messages=[
{"role": "system", "content": "You are a skilled mathematician & tasteful typographer, expert in LaTeX."},
{"role": "user", "content": prompt }
]
)
print(completion.choices[0].message.content)