The domain can be an interval, a ring of $N$ points, the positive reals under multiplication, the surface of a sphere. What every such domain carries is a symmetry, a group $G$ acting on $D$ under which $D$ looks the same from any of its points. The third column of the table below names it for each transform.

From Laplace in the 1780s to wavelets in the 1980s, the zoo is one idea repeated, and the idea comes from a first course in linear algebra.

A transform is the change of basis that makes the symmetry of its domain act diagonally. The variety of transforms is the variety of domains and their symmetries, and the thread frays in one predictable order as the symmetry becomes less commutative: abelian group, then Gelfand pair, then non-abelian group, then an equivariant map, where no diagonal is left at all.

The table is in the order of history, and its last column assigns each row to one of the four stages of the sentence. The sections after the table take the stages in order. Every other transform in the zoo is a child of one of these twelve rows and is listed, in the same columns, in a table at the end of its parent’s stage.

The transforms in chronological order

What a transform does

The idea is diagonalization. A unitary change of basis $A \mapsto U A U^{\ast}$ brings every normal operator to diagonal form; that is the spectral theorem.⁹ An integral transform is the $U$. Let $\tau_a$ be translation by $a$ on the real line, $(\tau_a f)(t) = f(t - a)$, and $\mathcal{F}$ the Fourier transform. Then

A multiplication operator rescales its input point by point, as a diagonal matrix rescales each coordinate. So $\mathcal{F}$ makes translation diagonal, and with it everything that commutes with translation, since commuting normal operators share an eigenbasis: $d/dt$ becomes multiplication by $i\xi$, and every convolution becomes multiplication by the transform of its kernel.⁹ This is what a transform is used for. A differential equation with constant coefficients becomes algebraic, and its solution comes back as a convolution of the data with a fixed kernel; the Coulomb potential and the heat kernel are the two classical cases, both found before the transform that explains them.¹⁰

What “diagonal” means. One point has to be fixed before the groups get bigger, because the matrix picture misleads in one respect. Translation on $\mathbb{R}$ has no eigenvectors in $L^2$: $e^{i\xi t}$ is bounded but not square-integrable. So there is no eigenbasis, and yet the display above is exact. What it says is that $\mathcal{F}$ carries $\tau_a$ to a multiplication operator, and that is the form the spectral theorem takes on an infinite-dimensional space: a normal operator is unitarily equivalent to multiplication by a function on some $L^2(\mu)$, a diagonal matrix being the case where $\mu$ is counting measure on $N$ points (multiplication-operator form). Throughout this post “diagonal” means this: after the transform, the operator is a multiplication. An eigenbasis exists when the spectrum is discrete, on a finite group, where the transform is an $N \times N$ unitary matrix and the first course applies verbatim, and on a compact group; on $\mathbb{R}$ the eigenfunctions $e^{i\xi t}$ exist but are not vectors of the space, and the diagonal is the multiplication.

Stage one: abelian groups

The setting. $G$ is a locally compact abelian group with Haar measure $dg$; when $G$ is finite, $dg$ is counting measure and every integral below is a sum. $L^2(G)$ is the space of square-integrable functions on $G$ with $\langle f, h \rangle = \int_G f \bar h\, dg$; for finite $G$ it is the space of all functions $G \to \mathbb{C}$. Translation by $g$ is $(\lambda(g) f)(x) = f(g^{-1} x)$, so that on $\mathbb{R}$, $\lambda(a) = \tau_a$; and convolution is $(f \ast h)(x) = \int_G f(g)\, h(g^{-1}x)\, dg$. The operators on $L^2(G)$ that commute with every translation are the convolutions $f \mapsto f \ast h$.¹¹

What breaks. Nothing yet. This is the stage the sentence describes verbatim, and the other three are measured against it.

What replaces the character. Nothing needs to; this is the stage that has them. The convolutions commute with one another, so one change of basis makes them all multiplications, and the common eigenfunctions are the characters, the continuous homomorphisms $\chi\colon G \to \mathbb{T}$: $\chi \ast h = \hat h(\chi)\, \chi$, where $\hat h(\chi)$ is the transform below.¹² The characters are bounded functions, in $L^2(G)$ only when $G$ is compact, which is the point of the paragraph on what “diagonal” means. The group decides the kernel, and the kernel column of the table is the characters written out. In symbols,

The characters form the dual group $\hat G$, and the transform is an isometry $L^2(G) \to L^2(\hat G)$, the Plancherel theorem, carrying $f \mapsto f \ast h$ to multiplication by $\hat h$: that is the diagonalization, in the sense fixed above, and the second half of the display is its formula. Further, $\hat{\hat G} = G$ (Pontryagin duality), and in the language of Banach algebras the transform is the Gelfand transform of $L^1(G)$.¹²

DFT. $G = \mathbb Z_N$, self-dual, with characters $\chi_k(j) = e^{2\pi i jk/N}$. The transform is the $N \times N$ unitary matrix $N^{-1/2}(e^{-2\pi i jk/N})_{k,j}$, and the first course applies without change.

Fourier series. $G = \mathbb{T}$, with dual $\mathbb{Z}$ and characters $e^{in\theta}$; the group is compact and the spectrum is discrete. The transform in the other direction, on $G = \mathbb{Z}$ with dual $\mathbb{T}$, is the Fourier series read backwards, $\hat f(e^{i\theta}) = \sum_n f(n) e^{-in\theta}$.³

Fourier transform. $G = \mathbb{R}$, self-dual, with characters $e^{i\omega t}$; the group is neither compact nor discrete, and neither is the spectrum: there is no eigenbasis, and the transform diagonalizes translation in the multiplication sense only.

Mellin. $G = (\mathbb{R}^+, \times)$. The substitution $t = e^{\xi}$ carries it to $(\mathbb{R}, +)$ and the kernel to $t^{s-1}\,dt = e^{s\xi}\,d\xi$, so the Mellin transform is the Fourier transform of $f(e^\xi)$ at the complex frequency $s$; the unitary characters are the $t^{i\omega}$, on the line $\Re s = 0$.⁵

Laplace. $G = \mathbb{R}$ on functions supported in $[0, \infty)$, with the character $e^{-i\omega t}$ continued to the complex frequency $s$, $\Re s > 0$. The gain is that $e^{-st}$ decays and the result is analytic in $s$; the cost is orthogonality, since the $e^{-st}$ for real $s$ are not orthogonal and there is no Plancherel formula on the real $s$-axis. The isometry holds line by line: on $\Re s = c$ the Laplace transform of $f$ is the Fourier transform of $e^{-ct} f$.¹³

Children of stage one

Changing the field: Hartley and NTT

Two of the children keep $G = \mathbb Z_N$ and change the field of scalars from $\mathbb{C}$ to a field $\mathbb{F}$. Over $\mathbb{F}$ the circulant matrices are $\mathbb{F}[x]/(x^N-1)$, the cyclic shift is the companion matrix of $x^N - 1$, and the Frobenius normal form says how far the shift can be reduced: to one block for each irreducible factor of $x^N - 1$ over $\mathbb{F}$.

The middle row is the real spectral theorem: no real basis separates a conjugate pair of eigenvalues, so the Hartley kernel $\cos\nu x + \sin\nu x$ pairs $\nu$ with $-\nu$, and no real transform does better.⁹ The last row says that a change of field costs nothing provided $\mathbb F_q^\times$, a cyclic group of order $q - 1$, contains an element of order $N$.

Stage two: Gelfand pairs

The setting. As in stage one, except that $G$ is a locally compact group, not necessarily abelian, and the domain is not $G$ but a space $X$ on which $G$ acts transitively. $K$ is the stabilizer of a chosen base point, so that $X = G/K$ is a homogeneous space: $S^2 = SO(3)/SO(2)$, and $\mathbb{R}^n = G/K$ with $G$ the Euclidean motion group and $K = SO(n)$, so that the radial functions are the $K$-invariant functions on $X$. $G$ acts on $L^2(X)$ by the formula of stage one, $(\lambda(g) f)(x) = f(g^{-1} x)$. A unitary representation $\pi$ of $G$ on a Hilbert space $V_\pi$ is irreducible when $V_\pi$ has no proper nonzero closed invariant subspace, and $d_\pi = \dim V_\pi$; a character is the case $d_\pi = 1$.

What breaks. The domain is no longer a group. The Hankel transform acts on radial functions on $\mathbb{R}^n$ and the spherical harmonic transform on $S^2$; the sphere cannot be made a group, and $SO(3)$, which acts on it, has no invariant one-dimensional space of functions but the constants.¹⁰ So there are no characters. Yet both transforms are diagonal: the Hankel transform turns the radial Laplacian into multiplication by $-\kappa^2$,⁴ and the spherical harmonics are eigenfunctions of the Laplacian on the sphere.²

What replaces the character. The operators on $L^2(X)$ commuting with every $\lambda(g)$ form an algebra, and $(G, K)$ is a Gelfand pair when that algebra is commutative; symmetric spaces are the motivating examples. When it is, three things hold at once. The commuting operators diagonalize simultaneously, as in stage one. $L^2(X)$ splits into irreducible pieces $V_\pi$, no piece repeated: a direct sum when $G$ is compact, as for $S^2$, where the spectrum is discrete, and with a continuous spectrum for $\mathbb{R}^n$, as on $\mathbb{R}$. And each $V_\pi$ contains, up to scale, one $K$-fixed unit vector $u_\pi$; read as a function on $X$ it is the spherical function of the piece, and it plays the part of the character:

The $\phi_\pi$ are the multiplicative functionals of the commutative algebra, exactly as the characters are for $L^1(G)$; and for $f \in L^2(X)$ not $K$-invariant, $\langle f, \pi(g) u_\pi\rangle$ as a function of $gK$ is a multiple of the projection of $f$ onto $V_\pi$.²¹ Stage one is the case $K = {1}$: $(G, {1})$ is a Gelfand pair if and only if $G$ is abelian, and the spherical functions are then the characters.²¹ The hypothesis was never commutativity of $G$ but commutativity of the operators commuting with $G$, equivalently no repeated piece. This is the first place the transforms need representation theory, and for one job: to say when a transform is still diagonal.

Spherical harmonic transform. $(G, K) = (SO(3), SO(2))$, $X = S^2$. The pieces $V_\ell$ are the spherical harmonics of degree $\ell$, the spherical function of $V_\ell$ is the Legendre polynomial $P_\ell(\cos\theta)$ with $\theta$ the angle from the fixed axis, and the transform of $f \in L^2(S^2)$ is its expansion in the $Y_\ell^m$; on zonal $f$ it is the integral against $P_\ell$.² For $(SO(n), SO(n-1))$ on $S^{n-1}$ the Legendre polynomials become Gegenbauer polynomials.²²

Hankel transform. $G = \mathbb{R}^n \rtimes SO(n)$, $K = SO(n)$, $X = \mathbb{R}^n$; the spectrum is continuous, so “diagonal” is again the multiplication sense. Here the commutativity is elementary: the operators commuting with the motion group are convolutions by radial kernels, and the Fourier transform makes them multiplications by radial functions.⁴ The Hankel transform of order $(n-2)/2$ is the Fourier transform on $\mathbb{R}^n$ restricted to radial functions, which is where the Bessel function in the kernel comes from. The other orders in the table come from the other pieces of $L^2(\mathbb{R}^n)$: on $f_0(\lvert x \rvert) P(x)$ with $P$ a harmonic polynomial of degree $k$, the Fourier transform is $P$ times a Hankel transform of order $k + (n-2)/2$.²

Children of stage two

Stage three: non-abelian groups

The setting. As in stage two, except that the domain is the group again, $X = G$ and $K = {1}$, with $G$ finite or a compact Lie group; $\lambda$ and $\ast$ are those of stage one, $\pi$, $V_\pi$, $d_\pi$ those of stage two, and $\chi_\pi = \mathrm{tr}\pi$ is the character of $\pi$, a function on $G$.

What breaks. $G$ is not commutative, so $(G, {1})$ is not a Gelfand pair: the convolutions $f \mapsto f \ast h$ do not commute with one another and no single eigenbasis exists.

What replaces the character. An irreducible representation $\pi$ of dimension $d_\pi$. The transform of $f$ at $\pi$ is a $d_\pi \times d_\pi$ matrix, and convolution becomes matrix multiplication, block by block:

the bar being the entrywise conjugate, so that the kernel is the $\overline{\pi_{ij}(g)}$ of the table.²³ For finite $G$ the statement that this is an isomorphism of algebras, $L^2(G) \cong \bigoplus_\pi M_{d_\pi}(\mathbb{C})$, is Wedderburn’s theorem; stage one is the case where every block is $1 \times 1$.²³ Block-diagonal is not diagonal, and the following observation is what makes the transform a diagonalization after all. Let $G \times G$ act on $L^2(G)$ by translating on both sides, $f(g) \mapsto f(g_1^{-1} g g_2)$. Under this larger action the decomposition $L^2(G) = \bigoplus_\pi V_\pi \otimes V_\pi^{\ast}$ has no repeated piece, so $(G \times G, \Delta G)$ with $\Delta G = {(g, g)}$ is a Gelfand pair, and its spherical functions are $\chi_\pi / d_\pi$; this is what “character” means for a non-abelian group.²³ So the non-abelian transform diagonalizes the two-sided action, and the $d_\pi \times d_\pi$ blocks are what the one-sided action alone can see. The history ran the same way: Frobenius invented representation theory in 1897 to answer a question of Dedekind’s about group determinants, and it took until Weyl, thirty years later, for anyone to see that he had extended Fourier analysis from commutative to non-commutative groups.⁶

Fourier transform on a finite group. $G$ finite, the integral a sum. The entries $\pi_{ij}$, over all $\pi$ and all $i, j \le d_\pi$, are an orthogonal basis of $L^2(G)$, and $\lvert G \rvert = \sum_\pi d_\pi^2$; the transform is the unitary $\lvert G \rvert \times \lvert G \rvert$ matrix that expands $f$ in this basis.²³

Peter–Weyl transform. $G$ a compact Lie group such as $SO(3)$ or $SU(2)$, the integral against Haar measure. The same statement holds with countably many $\pi$: the matrix coefficients $\pi_{ij}$ are a complete orthogonal system in $L^2(G)$, which is the Peter–Weyl theorem of 1927.⁶

Cayley graphs and shuffles

Two uses of the block form. A Cayley graph on $G$ has adjacency matrix equal to convolution by the indicator $\delta_S$ of its generating set $S$, so its eigenvalues are those of the blocks $\hat\delta_S(\pi)$, each repeated $d_\pi$ times; when $G$ is abelian, or $S$ is a union of conjugacy classes, the blocks are scalars and the eigenvalues are the character sums $\frac{1}{d_\pi}\sum_{s \in S} \chi_\pi(s)$.²⁴ And a shuffle is a probability $P$ on $S_n$, $k$ shuffles is $P^{\ast k}$, so $\widehat{P^{\ast k}} = \hat P^k$, and Plancherel bounds the total-variation distance $\lVert P^{\ast k} - U \rVert = \max_A \lvert P^{\ast k}(A) - U(A) \rvert$ from the uniform distribution $U$ by a sum over representations:

For $\mathbb Z_p$, stepping by $\pm 1$, somewhat more than $p^2$ steps are needed. For $S_n$, swapping a random pair each step, $\frac12 n\log n + cn$ shuffles bring the distance below $a e^{-2c}$, while $\frac12 n \log n - cn$ leave it above $\frac1e - o(1)$: the distance drops from a constant to nearly zero in a window of width $O(n)$ around $\frac12 n \log n$, which is the cutoff phenomenon.²⁴

Stage four: equivariant maps

The setting. As in stage two, except that the transform now goes between two different spaces on which $G$ acts. For the Radon transform they are two homogeneous spaces, $X = G/K$ as before and a second one $Y = G/H$, with $H$ the stabilizer of a base point of $Y$; $\lambda_X$ and $\lambda_Y$ are the actions of $G$ on $L^2(X)$ and $L^2(Y)$ by the formula of stage one. For the wavelet and Gabor transforms the source is $L^2(\mathbb{R})$ with $G$ acting by a unitary representation $\pi$, and the target is $L^2(G)$ with the action $\lambda$ of stage one.

What breaks. The diagonal. In the three stages so far the transform was a change of basis in which the symmetry acts by scalars, or at worst by blocks; in the three remaining rows the group is still there but the target of the map is not a sum of one-dimensional pieces, and there are no scalars to be found.

What replaces the character. Nothing does; what survives is the property that made the character work. A transform $T$ from one of the two spaces to the other is an equivariant map, one that commutes with the two actions:

Stage one is the case $X = G$, $Y = \hat G$, where $\lambda_Y(g)$ is multiplication by $\overline{\chi(g)}$, a scalar at each point of $Y$.¹² The kernel of an equivariant map is still decided by the group, as it was there: the equivariant maps $L^2(G/K) \to L^2(G/H)$ correspond to the functions $\kappa$ on $G$ with $\kappa(kgh) = \kappa(g)$ for $k \in K$, $h \in H$, and that is why these three are still called transforms.⁷

Radon transform. $X$ is the set of points of the plane and $Y$ the set of lines; the group $G$ of rigid motions acts transitively on both, with $K$ the stabilizer of a point and $H$ that of a line. The Radon transform integrates $f$ over each line, and integrating over a line commutes with moving the plane, so it is equivariant; the function $\kappa$ is the incidence relation, $1$ when the point lies on the line. The finite version, with $S_n$ acting on $j$-subsets and $k$-subsets of ${1, \dots, n}$ and $\kappa$ the containment relation, is where this description is proved and inverted.⁷

Wavelet transform. $G$ is the affine group ${t \mapsto at + b : a > 0}$, acting on $L^2(\mathbb{R})$ by $(\pi(a,b) f)(t) = a^{-1/2} f((t-b)/a)$, and the transform pairs $f$ with the translates of a chosen window $\psi$:

where $\lambda$ is translation on functions of $(a,b)$, as in stage one. So $W_\psi$ is an equivariant map from $L^2(\mathbb{R})$ into functions on $G$. What survives of the diagonal is energy: an admissibility condition on $\psi$, $C_\psi = 2\pi\int \lvert \hat\psi(\omega) \rvert^2 \lvert \omega \rvert^{-1} d\omega < \infty$, makes $C_\psi^{-1/2} W_\psi$ an isometry into $L^2$ of the half-plane with the measure $db\,da/a^2$.⁸ Compare stage two, where the transform paired $f$ with the translates $\pi(g) u_\pi$ of the one $K$-fixed vector: here $\psi$ is fixed by no stabilizer at all. Dropping that invariance costs the diagonal, since there is no commutative algebra left to diagonalize, and leaves the isometry, which never depended on characters.

Gabor transform. The same construction with a window $g$ translated in time and modulated in frequency, $g_{b,\omega}(t) = g(t-b)\, e^{i\omega t}$: the transform is $\langle f, g_{b,\omega}\rangle$, the kernel in the table is $\overline{g_{b,\omega}}$, and time shifts and modulations of $f$ shift the transform in $(b, \omega)$.²⁵

Children of stage four

The four canonical forms

Classifying the elements of a matrix group up to conjugacy is the problem of canonical forms, and the four forms of a first course are all the post has used.²⁴

Where the thread ends

Two things that carry the name “transform”, or are treated alongside them, are outside the four stages, each for a different reason.

Operators a transform diagonalizes. The Hilbert transform is convolution with $1/\pi t$ on the line, so it commutes with translation and the Fourier transform diagonalizes it, as multiplication by $-i \mathrm{sgn} \xi$.²⁶ The Stieltjes transform is the Laplace transform applied twice, $\int_0^\infty f(t)\,dt/(t+x)$; its kernel depends on $t$ and $x$ only through $t/x$, up to the factor $1/x$, so it commutes with dilations and the Mellin transform diagonalizes it, as multiplication by $\pi/\sin \pi p$.²⁷ The Weyl fractional integral is of the same kind, a Mellin multiplier.²⁷ These are not changes of basis but the kind of operator a change of basis is for, and that is why they have no row.

No basis at all. Let $A \subset \mathbb Z_N$ have density $\alpha$. The fraction of three-term arithmetic progressions $x + y = 2z$ with all three terms in $A$ is $\sum_r \hat A(r)^2 \hat A(-2r)$, which is $\alpha^3$, the random share, plus an error of at most $\alpha \max_{r \ne 0} \lvert \hat A(r) \rvert$; so either $A$ has its random share or a character correlates with it, and iterating on the progressions where that character is nearly constant is Roth’s theorem of 1953.²⁸ The Gowers norms (1998) are where this stops. On $\mathbb Z_N$ with $N$ odd the function $e^{2\pi i x^2/N}$ has every Fourier coefficient of modulus $N^{-1/2}$, as flat as Plancherel allows, yet it and three companions have four-term progression count $1$, the largest possible: the transform is blind to the quadratic phases. There are $N^2$ of them against $N$ characters, so they are no basis, and no quadratic transform or inversion formula exists. What survives is the $U^3$ norm, an average over cubes, and an inverse theorem: a function with large $U^3$ norm correlates with a phase built from a two-step nilpotent group, a Heisenberg group once more.²⁹ Nothing is being diagonalized, so the thread does not fray here; it ends.

One idea from a first course in linear algebra reaches across two and a half centuries into number theory, tomography, image compression and quantum computation. Representation theory is what one reaches for where it stops, and it works there because diagonalization was one of its tools and never the whole of it.

Sources

An AI slop. Brought to you by Anthropic Claude Opus 5.

• Lokenath Debnath and Dambaru Bhatta, Integral Transforms and Their Applications, 3rd ed., CRC Press, 2015.
• Audrey Terras, Fourier Analysis on Finite Groups and Applications, LMS Student Texts 43, Cambridge, 1999.
• Walter Rudin, Fourier Analysis on Groups, Interscience, 1962.
• Elias M. Stein and Guido Weiss, Introduction to Fourier Analysis on Euclidean Spaces, Princeton Mathematical Series 32, Princeton, 1971.
• Paul R. Halmos, Finite-Dimensional Vector Spaces, 2nd ed., Springer UTM.
• Benjamin Steinberg, Representation Theory of Finite Groups: An Introductory Approach, Springer Universitext, 2012.
• Tullio Ceccherini-Silberstein, Fabio Scarabotti and Filippo Tolli, Harmonic Analysis on Finite Groups, Cambridge, 2008 (cited as CST 2008).
• Tullio Ceccherini-Silberstein, Fabio Scarabotti and Filippo Tolli, Discrete Harmonic Analysis, Cambridge, 2018 (CST 2018).
• Persi Diaconis, Group Representations in Probability and Statistics, IMS Lecture Notes 11, 1988.
• David A. Levin and Yuval Peres, with Elizabeth L. Wilmer, Markov Chains and Mixing Times, 2nd ed., AMS, 2017 (LPW).
• Michael A. Nielsen and Isaac L. Chuang, Quantum Computation and Quantum Information, Cambridge.
• George W. Mackey, Harmonic analysis as the exploitation of symmetry: a historical survey, Bull. AMS 3 (1980), 543–698.
• Louis Auslander and Richard Tolimieri, Is computing with the finite Fourier transform pure or applied mathematics?, Bull. AMS (N.S.) 1 (1979), no. 6, 847–897 (cited as AT).
• W. T. Gowers, Generalizations of Fourier analysis, and how to apply them, Bull. AMS (N.S.) 54 (2017), no. 1, 1–44.

The advent of coding agents like Claude Code, Qwen Code, Cursor, Windsurf, and Antigravity seems to pose a similar threat to the careers of computer programmers. Hence, to me, it feels like history is repeating itself. The era of the coder is evolving just as the human computer did. It is now time for programmers to pivot toward roles that go beyond writing code and instead focus on the high-level problem-solving and logic seen in competitive programming. So, I asked Google Gemini to list similar professions so that we can learn from history, especially the kind of pivots that worked.

*Quantum navigation is replacing GPS.

On the other hand, the rise of Google Search revolutionized information access, fundamentally transforming the role of the traditional librarian. While physical foot traffic for basic fact-finding declined as knowledge became digitized, the profession evolved into Library and Information Science (LIS), focusing on digital curation and information literacy. In the corporate world, this analytical shift led many into Data Science. However, as generative AI tools like Gemini, ChatGPT, and Perplexity begin to automate routine data cleaning and statistical testing, the data scientist’s role is shifting once again. To remain indispensable, they must pivot beyond basic execution toward advanced statistical validation, algorithmic oversight, and the complex interpretation of AI-generated insights.

In 2024, the Nobel Prizes in Physics and Chemistry were both awarded for breakthroughs directly tied to Artificial Intelligence. John Hopfield and Geoffrey Hinton received the Physics prize for the foundational neural network discoveries that power modern AI, while the Chemistry prize was awarded to the creators of AlphaFold for using AI to solve the 50-year-old challenge of protein structure prediction. This shift signals that AI has transitioned from a niche experiment to an essential scientific instrument. Consequently, researchers who fail to integrate these tools into their workflows risk becoming obsolete in an increasingly automated landscape.

As a mathematician, I also feel the pressure mounting. For example, celebrated mathematicians like Terence Tao, Jordan Ellenberg, and Ken Ono have jumped headfirst into neural theorem proving. However, history suggests this maybe an evolution rather than an end. Chess engines became training grounds for chess players instead of killing the sport. Similarly, it will be interesting to see the impact of AI agents on mathematics competitions and research.

Over time, I have developed a sense of what I want and am ready to be responsible for my Thinkpad P16 :)

I learned about the process from the official guide and supplemented it with the notes by Michele Gementi, Max Pershin, Abhishek Prakash, quantinium, Daniel Wayne Armstrong, Gentoo and Ubuntu. It took around half-dozen test runs in VirtualBox over the course of a month before performing the actual installation. Here is a documentation to get started with the best operating system.

I will follow the Arch Wiki convention: The numeral or hash sign (#) indicates that the command needs to be run as root, whereas the dollar sign ($) shows that the command should be run as a regular user.

Prepararation

1. Check ArchLinux website status since they are prone to DDoS attacks. I encountered this during my first ever test run of Arch Linux. Also, the services may send an initial connection reset due to the TCP SYN authentication performed by the hosting provider, but subsequent requests should work as expected.
   1. In the case of downtime for archlinux.org:
      1. Mirrors: The mirror list endpoint used in tools like reflector is hosted on this site. Please default to the mirrors listed in the pacman-mirrorlist package during an outage.
      2. ISO: Installation image is available on a lot of the mirrors, for example the DevOps administered geomirrors. Please always verify its integrity.
   2. In the case of downtime for aur.archlinux.org:
      1. Packages: A mirror of AUR packages is maintained on GitHub. You can retrieve a package using:

          $ git clone --branch <package_name> --single-branch https://github.com/archlinux/aur.git <package_name>
         

      2. I will not use any package from AUR in this tutorial. It is similar (but superior) to the third-party repositories like PPA for Ubuntu, COPR for Fedora, and OBS for openSUSE.
2. Download the latest archlinux-YYYY.MM.DD-x86_64.iso, sha256sums.txt, and archlinux-YYYY.MM.DD-x86_64.iso.sig from the nearest mirror like mirror.arizona.edu.
3. Verify the integrity and authenticity of your ISO image using Linux (if Windows, use WSL).
   1. Integrity: Ensure the download image matches the checksum from the sha256sums.txt

       $ sha256sum -c sha256sums.txt
      

      This should output archlinux-YYYY.MM.DD-x86_64.iso: OK. You may see “No such file or directory” for other files which we didn’t download but their checksums also included.

   2. Authenticity: One should never use a GnuPG version just downloaded from internet to check the integrity; instead use an existing, trusted GnuPG installation, e.g., the one provided by your Linux distribution (if Windows, then use WSL).

       $ gpg --auto-key-locate clear,wkd -v --locate-external-key pierre@archlinux.org
       $ gpg --verify archlinux-YYYY.MM.DD-x86_64.iso.sig archlinux-YYYY.MM.DD-x86_64.iso
      

      GPG might give warning This key is not certified with a trusted signature!, but you can verify it here.

4. Create a bootable USB drive using Ventoy; remember to verify the checksum of Ventoy itself.
5. In UEFI, disable secure boot.

From chroot to root

1. Boot from the Live USB drive and select Arch Linux from the systemd-boot greeting screen to get logged in on the first virtual console as the root user, and presented with a Zsh shell prompt.
2. Verify that the boot mode is UEFI by ensuring that the following command returns 64.

    # cat /sys/firmware/efi/fw_platform_size
   

3. Connect to WiFi using iwctl.

    # iwctl
    [iwd]# device list
    [iwd]# station <device-name> scan
    [iwd]# station <device-name> get-networks
    [iwd]# station <device-name> connect <Name of WiFi access point>
    [iwd]# exit
    $ ping -c 5 ping.archlinux.org
   

4. Ensure that the system clock is syncronized via Network Time Protocol (NTP).

    # timedatectl set-ntp true
    # timedatectl
   

5. Use cfdisk to create the following GUID Partition Table (GPT), without hibernation feature (suspend-to-disk).

   Mount point	Parition type	Size	Partition
   /efi	EFI System Partition	1 GiB	/dev/efi_system_partition
   [swap]	Linux swap	6 GiB	/dev/swap_partition
   /	Linux Filesystem	(all of the remaining space )	/dev/root_partition

   Can later get the partition names using fdisk -l.

6. Format the partitions using mkfs.

    # mkfs.fat -F 32 /dev/efi_system_partition
    # mkswap /dev/swap_partition
    # mkfs.btrfs /dev/root_partition
   

7. Create the Btrfs filesystem layout recommended for snapper

    # mount /dev/root_partition /mnt
    # btrfs subvolume create /mnt/@
    # btrfs subvolume create /mnt/@home
    # btrfs subvolume create /mnt/@var_log
    # btrfs subvolume create /mnt/@var_cache
    # btrfs subvolume create /mnt/@var_spool
    # btrfs subvolume create /mnt/@var_tmp
    # btrfs subvolume create /mnt/@snapshots
    # btrfs subvolume list -t /mnt
    # lsblk --fs
    # umount /mnt
   

8. Mount the file systems as we prepare to chroot with Btrfs compression settings optimized for NVMe SSD (Kioxia XG8) and not using Mutt.

    # mount -o noatime,compress=zstd:1,subvol=@ /dev/root_partition /mnt
    # mkdir -p /mnt/{home,.snapshots} /mnt/var/{log,cache,spool,tmp}
    # mount -o noatime,compress=zstd:1,subvol=@home /dev/root_partition /mnt/home
    # mount -o noatime,compress=zstd:1,subvol=@var_log /dev/root_partition /mnt/var/log
    # mount -o noatime,compress=zstd:1,subvol=@var_cache /dev/root_partition /mnt/var/cache
    # mount -o noatime,compress=zstd:1,subvol=@var_spool /dev/root_partition /mnt/var/spool
    # mount -o noatime,compress=zstd:1,subvol=@var_tmp /dev/root_partition /mnt/var/tmp
    # mount -o noatime,compress=zstd:1,subvol=@snapshots /dev/root_partition /mnt/.snapshots
    # mount --mkdir /dev/efi_system_partition /mnt/efi
    # swapon /dev/swap_partition
    # lsblk --fs
   

9. Use pacstrap to create a new system installation from scratch.

    # pacstrap -K /mnt
   

   This installs the base metapackage consisting of basic tools like bash, systemd, and pacman. It also copies the LiveUSB’s mirrorlist /etc/pacman.d/mirrorlist generated using reflector to the new system. During the fifth test run I was bit by a bug in this step.

10. Generate fstab file with startup instructions

    # genfstab -U /mnt >> /mnt/etc/fstab
    

    Check the resulting /mnt/etc/fstab file, and edit it in case of errors and various mount options like discard=async and ssd.

11. Switch to the new system’s environment

    # arch-chroot /mnt
    

    1. Set up the time zone.

        # ls /usr/share/zoneinfo/                                     
        # ls /usr/share/zoneinfo/<Zone>/                              
        # ln -sf /usr/share/zoneinfo/<Zone>/<Subzone> /etc/localtime       
        # hwclock --systohc
       

    2. Configure correct region and language specific formatting.
       1. Install a console text-editors like nano:

          # pacman -S nano
          

       2. Use nano to edit /etc/locale.gen and uncomment en_US.UTF-8 UTF-8.
       3. Generate the locales.

          # locale-gen
          

       4. Use nano to create configuration file /etc/locale.conf and write LANG=en_US.UTF-8 in it.
    3. Network configuration.
       1. Use nano to create /etc/hostname and write <PC-Name> in it.
       2. Use nano to edit /etc/hosts and add the following to it

          127.0.0.1 localhost
          ::1 localhost
          127.0.1.1 <PC-Name> 
          

       3. Install and enable network manager to run on boot.

          # pacman -S networkmanager
          # systemctl enable NetworkManager
          

    4. Create a new initramfs (initial RAM file system)
       1. Install userspace utilities for the Btrfs file system (btrfs-progs) and Linux kernel linux with linux-lts as a fallback option. If presented with options for initramfs, choose mkinitcpio. Note that trying to install kernel without btrfs-progs will lead to error when mkinitcpio runs the fsck hook.
       2. Install kernel firmware (linux-firmware) along with additional firmware for Intel CPU (intel-ucode), Intel iGPU (mesa, vulkan-intel), NVIDIA dGPU (nvidia, nvidia-utils), and Dolby Atmos (pipewire, pipewire-alsa, pipewire-pulse, pipewire-jack, wireplumber). Update (Dec 20, 2025): Use nvidia-open instead of nvidia, see news.
       3. Re-run mkinitcpio to be sure that we have an error free initial ramdisk environment.

          # mkinitcpio -P
          

          Note that every time a kernel is installed or upgraded, a pacman hook automatically generates a preset file saved in /etc/mkinitcpio.d/ and -P option process all presets contained in /etc/mkinitcpio.d.

    5. Set root password using passwd command.
    6. Install a btrfs-friendly bootloader, like GRUB.

        # pacman -S grub efibootmgr
        # grub-install --target=x86_64-efi --efi-directory=/efi --bootloader-id=GRUB
        # grub-mkconfig -o /boot/grub/grub.cfg
       

       Kernel packages are installed under the /usr/lib/modules/ path and subsequently used to copy the vmlinuz executable image to /boot/.

    7. We can use the bootloader to switch between linux and linux-lts kernel. But will set linux as the default option by editing /etc/default/grub:

       GRUB_DISABLE_SUBMENU=y
       GRUB_DEFAULT="Arch Linux, with Linux linux"
       

       Regenerate your configuration file with grub-mkconfig.

    8. Exit the chroot environment by typing exit or pressing Ctrl + d.
12. Manually unmount all the partitions

    # umount -R /mnt
    

13. Reboot (or poweroff) the system and remove the installation media

    # reboot
    

From superuser to user

1. Login into the new system with the root account (superuser).
2. Connect to WiFi using nmtui or nmcli.

    # nmcli device wifi connect <Name of WiFi access point> password <password>
   

3. Pacman configuration to automatically update mirrors and periodically clear out cache.

    # pacman -S pacman-contrib
    # systemctl enable --now paccache.timer
    # pacman -S reflector
    # systemctl enable --now reflector.timer
   

4. Install man to have offline access to Arch Linx documentation.

    # pacman -S man-db
   

5. Create a user and add to administration group (wheel) with sudo access. Note that logging in using root account is disabled for GUI.

     # useradd -mG wheel <user-name>
     # passwd <user-name>
     # pacman -S sudo
     # EDITOR=nano visudo
   

   The last command opens the /etc/sudoers file using nano; look for a line which says something like Uncomment to allow members of group wheel to execute any command and uncomment exactly the line BELOW it, by removing the #. This will grant superuser priviledges to your user.

6. Check NVIDIA configuration by ensuring that the Direct Rendering Manager (DRM) is enabled.

    # cat /sys/module/nvidia_drm/parameters/modeset
   

   This should return Y.

7. Install KDE with Wayland support following advice from the KDE documentation and Arch dependency tree of plasma-desktop. We have consciously excluded discover from KDE because it can lead to partial upgrades which are unsupported in Arch.
   1. base: plasma-{desktop,pa,nm,systemmonitor,firewall}, kscreen, bluedevil, powerdevil, tlp
   2. keyring (PAM): kwalletmanager, kwallet-pam
   3. terminal (otherwise will need to use tty): konsole
   4. file manager: dolphin, dolphin-plugins, kdegraphics-thumbnailers, ffmpegthumbs
   5. XDG Desktop Portal (Wayland and Firefox integration): xdg-desktop-portal-gtk, xdg-desktop-portal-kde
   6. Theme consistency (GTK in Qt): breeze-gtk, kde-gtk-config

       # pacman -S <space separated list of packages>    
      

      You may be prompted to choose between ffmpeg or gstreamer backend for qt6-multimedia. I chose ffmpeg because that seems to be the default.

8. Install Simple Desktop Display Manager (SDDM).

    # pacman -S sddm
    # systemctl enable sddm
    # pacman -S --needed sddm-kcm
   

9. Reboot to finalize installation.

    # reboot
   

Personalization

1. Login using user account in SDDM.
2. Check if any systemd services have failed or errors in the log files located in /var/log/.

    $ systemctl --failed
    $ journalctl -b -p 3
   

3. Go to Settings → Colors & Themes → Login Screen (SDDM) → Breeze → Change Background → Apply.
4. Install essential software using the package manager.
   1. Browser: firefox (along with extensions: uBlock Origin, Privacy Badger, Decentraleyes, ClearURLs, and password manager)
   2. Document viewer: okular, ebook-tools, kdegraphics-mobipocket, unarchiver
   3. Image viewer: gwenview, kimageformats, qt6-imageformats
   4. Video player (mpv and yt-dlp based): haruna, yt-dlp
   5. Archiving tool: ark, 7zip
   6. Screenshot tool: spectacle
   7. Calculator: kalk
   8. Text editor: kate
   9. File searching tool: kfind
   10. Messaging: signal-desktop
   11. GPU manager (NVIDIA): nvidia-settings
   12. Webcam controls (LogiTune): cameractrls
   13. Wireless mouse (LogiBolt): solaar
   14. IDE: emacs-wayland
5. Battery care configuration using tlp.

   $ sudo systemctl enable --now tlp.service
   $ sudo tlp-stat -s
   $ sudo tlp-stat -b
   

   We can configure tlp by editing battery care settings in /etc/tlp.conf.

6. Configure Git and GitHub.

    $ sudo pacman -S git openssh
   

   1. Set the name and email for git commits. Also set default editor and push behavior.

      $ git config --global user.name  "Your Name"
      $ git config --global user.email "your.email@github.com"
      $ git config --global core.editor "nano -w"
      $ git config --global push.default simple
      

   2. Configure SSH for GitHub interaction:
      1. Generate a new public and private SSH key pair using OpenSSH.
      2. Add SSH public key to the GitHub account.
7. Configure rclone for Google Drive backup.

    $ sudo pacman -S rclone
   

   1. Create your own Google Drive OAuth2 client ID for rclone:
      1. Log into the Google API Console with your Google account. It doesn’t matter what Google account you use. (It need not be the same account as the Google Drive you want to access.
      2. Select a project or create a new project.
      3. Under “ENABLE APIS AND SERVICES” search for “Drive”, and enable the “Google Drive API”.
      4. Click “Oauth Consent Screen” in the left panel and select user type “External”. Then add Application name (anything you want) and save.
      5. Click “Credentials” in the left panel. Then click on “+ CREATE CREDENTIALS” button at the top of the screen, then select “OAuth client ID”. Select Application type as “Desktop app”, enter whaever client anme you want and click create.
      6. It will show you a client ID and client secret. Use these values in rclone config.
   2. Now configure rclone for Google Drive via Terminal: rclone config and follow the steps. Remember to use the Client ID and client secret we created above. Since we created this API for personal use, we won’t be submitting it for verfication. Hence don’t be alarmed by the very scary confirmation screen shown when we connect via your browser for rclone to be able to get its token-id. Also, if you want to fetch Google Docs as links (instead of converting them to .odt etc) from Google Drive, in “advance-config” set “export-formats” to “link.html”.
   3. Sync files using “copy” and NOT “sync”: rclone copy source:path dest:path [flags]. For example, to sync all files from “New Folder” Google Drive (named: Drive) to PC (folder: home) and view the progress, type: rclone copy Drive:"New Folder" /home -P. Google Drive tend to have duplicate files since it allows same names files in same folder, in that case use dedupe to delete all duplicate files.
8. Configure snapper for automatic / snapshots per the Wiki.

    $ sudo su
    # pacman -S snapper
   

   1. Unmount /.snapshots

      # btrfs subvolume list -t /
      # lsblk --fs
      # umount /.snapshots
      # rm -r /.snapshots
      

   2. Create a new snapper configuration named root for the Btrfs subvolume at /.

       # snapper -c root create-config /
       # btrfs subvolume list -t /
       # btrfs subvolume delete /.snapshots
       # mkdir /.snapshots
       # mount -o noatime,compress=zstd:1,subvol=@snapshots /dev/root_partition /.snapshots
      

      This will create a configuration file at /etc/snapper/configs/root. When you delete config file, then also delete its entry from /etc/conf.d/snapper to read SNAPPER_CONFIGS="".

   3. Make this mount permanent by adding an entry to fstab.

       # btrfs subvolume list -t /
       # lsblk --fs
       # blkid
       # nano /etc/fstab
       # mount -a
       # chmod 750 /.snapshots
       # chown :wheel /.snapshots
      

   4. Use systemd timer units for automatic timeline snapshots and cleanup.

       # systemctl enable --now snapper-timeline.timer
       # systemctl enable --now snapper-cleanup.timer
       # systemctl edit --full snapper-timeline.timer
       # systemctl edit --full snapper-cleanup.timer
      

      For taking snapshots use realtime timer OnCalendar=hourly in snapper-timeline.timer unit and for cleanup use monotomic timer OnBootSec=10m and OnUnitActiveSec=1h in snapper-cleanup.timer unit. Unlike Timeshift, we will not use cron job.

   5. Set snapshot limits by editing /etc/snapper/configs/root

      # limit for number cleanup
      NUMBER_MIN_AGE="1800"
      NUMBER_LIMIT="10"
      NUMBER_LIMIT_IMPORTANT="5"
      
      # limits for timeline cleanup
      TIMELINE_MIN_AGE="1800"
      TIMELINE_LIMIT_HOURLY="5"
      TIMELINE_LIMIT_DAILY="7"
      TIMELINE_LIMIT_WEEKLY="0"
      TIMELINE_LIMIT_MONTHLY="0"
      TIMELINE_LIMIT_YEARLY="0"
      
      # limits for empty pre-post-pair cleanup
      EMPTY_PRE_POST_MIN_AGE="1800"
      

      Here 1800 seconds = 30 min.

   6. We can use the grub-btrfs daemon to automatically update GRUB upon snapshot creation or deletion, and avoid dependency on LiveUSB for restoring snapshots.

      # pacman -S grub-btrfs inotify-tools
      # systemctl enable --now grub-btrfsd
      

      Then enable booting into read-only snapshots using overlay filesystem by adding grub-btrfs-overlayfs to the end of the HOOKS array in /etc/mkinitcpio.conf and regenerating the initramfs.

      # nano /etc/mkinitcpio.conf
      # mkinitcpio -P
      

      If you try to boot from these read-only snapshots available in GRUB menu, the following warning/error might appear:

      ********************* WARNING *********************
      * The root device is not configured to be mounted *
      * read-write! It may be fsck'd again later.       *
      ***************************************************
      [FAILED] Failed to start Remount Root and Kernel File Systems.
      

      Here the WARNING just confirms that our snapshot is read-only and [FAILED] confirms that systemd failed to mount the underlying read-only root filesystem. Despite this “failure” message, the boot process should continue using OverlayFS’s temporary read-write upper layer, leaving the original Btrfs snapshot untouched.

   7. Use snap-pac to make pacman automatically use snapper to create pre/post snapshots.

      # pacman -S snap-pac
      

   8. Testing snapshots by installing inxi.

      # pacman -S inxi
      # inxi -Fxz
      # snapper -c root list
      

9. Reboot and check if things if the things are as expected.

Maintainance

1. Must subscribe to the mailing list is for official announcements and check that before updating.
2. When refreshing the package database, always do a full upgrade.

   # pacman -Syu
   

3. Uninstall pacakge along with its dependecies.

   # pacman -Rs <package>
   

4. Recursively remove orphaned packages that were installed as a dependency but now, no other packages depend on them.

   # pacman -Qdtq | pacman -Rns -
   

5. Check list all foreign packages that are no longer be in the remote repositories, but still on your local system.

   # use pacman -Qm.
   

   This list will also include packages that have been installed manually (e.g., from the AUR).

6. Use nvme to probe the health of SSD.

   # pacman -S nvme-cli 
   # nvme smart-log /dev/root_partition
   

7. To view a list of snapshots under root configurations.

   # snapper -c root list
   

8. If GUI fails then can use LiveUSB arch-chroot or Desktop environment’s tty.
9. To restore / using one of snapper’s snapshots, we need access to OverlayFS (either via LiveUSB or grub-btrfs) and follow these steps.
   1. Mount the toplevel subvolume.

      # fdisk -l
      # mount /dev/root_partition /mnt
      # lsblk --fs
      # btrfs subvolume list -t /mnt
      

   2. Move @ to another location (e.g. /@.broken) to save a copy of the current system.

      # mv /mnt/@ /mnt/@.broken
      # btrfs subvolume list -t /mnt
      

   3. Find the number of the snapshot that you want to recover:

      # grep -r '<date>' /mnt/@snapshots/*/info.xml
      

   4. Create a read-write snapshot of the read-only snapshot snapper took:

      # btrfs subvolume snapshot /mnt/@snapshots/<number>/snapshot /mnt/@
      

   5. Check fstab is correct.

      # cat /mnt/etc/fstab
      

   6. Unmount the top-level subvolume (ID=5), then mount @ to /mnt and your ESP or boot partition to the appropriate mount point.

      # umount /mnt
      # mount -o noatime,compress=zstd:1,subvol=@ /dev/root_partition /mnt
      # mount /dev/efi_system_partition /mnt/efi
      

   7. If using LiveUSB then Change root to your restored snapshot before regenerating initramfs image.

      # arch-chroot /mnt     
      # mkinitcpio -P
      # exit
      # reboot
      

   8. If everything is as expected then delete the broken snapshot.

      # btrfs subvolume delete /@.broken
      

10. Vendor firmware updates (like UEFI)

       # pacman -S fwupd udisks2
       # fwupdmgr get-devices
    

The past

In the past, I have dabbled in programming using different languages:

Unfortunately, I never spent time honing my programming skills. Therefore, at the beginning of my graduate school, I thought of properly learning programming in Python. However, most of the study materials I found were more about “writing code to solve data science problems” rather than “learning to solve problems using programming.”

The present

I spent one year learning the basics of computer science following the advice from UArizona Computer Science department students and professors. None of the textbooks were read cover to cover.

During this one-year journey, I got the opportunity to practice programming in different languages:

I also learned regular expression usage in AWK (Frequency Analysis) and Python (Lexical Analysis).

The future

Next, I would like to improve my programming skills by

• using Python to solve the exercises from Project Euler, id0-rsa, Cryptopals, CrytoHack, and other CTFs;
• using C++ to solve competitive programming problems from Advent of Code and popular online judges (eg. UVa, Kattis, Codeforces, Sphere, AtCoder, Timus, DMOJ, Marisa, …);
• contributing to Open Source projects, especially SageMath;

However, there are many more programming languages to choose from. For example, the following are some of the main programming languages developed before the Internet age (source: Programming Languages - Principles and Practice, by Louden & Lambert, 3rd ed)

Eventually, I would also like to learn Rust and contribute to modern computer security projects.

The Nexus 7 is one of the best piece of hardware I ever owned, but it lacked good quality acccesories. Moreover, in my opinion, Samsung Galaxy Tab S6 Lite is a modified version of Samsung Google Nexus 10 with added Wacom drawing tablet functionality, reduced screen resolution, and removed camera-flash (full comparison). On the other hand, Samsung Galaxy Tab S series in general have filled the void left by Google Pixel C after Google dumped Android in favor of ChromeOS tablet Google Pixel Slate which supported Pixelbook Pen (Wacom’s AAAA-battery AES pen).

Initially, since reading was the main purpose of the tablet, I tried replacing my older tablet with Kindle Paperwhite (3rd generation). However, its small screen made it impossible to quickly jump between the pages of the large documents. Moreover, to be able to read pdf/DjVu files properly, one first has to optimize them for viewing using k2pdfopt or jailbreak kindle (complicated). Another adavantage of using a tablet PC instead of an e-reader is that nowadays there are many open source online textbooks available, like The Stacks project and nLab.

Review

Took some time to get used to writing on bigger screen than Nexus 7.

Settings > Display > Navigation bar > Swipe gestures > Swipe from sides and bottom

Also, enable “Block gestures with S Pen” and in Xodo app enable “Stylus as Pen”

Cons: low resolution, speakers position only good for landscape viewing, front camera is only good for portrait use.

The free rein of proprietary softwares

In the world of (abstract) mathematical computations, proprietary softwares tend to be more advanced than the open source alternatives. This is mainly because of the lack of industrial support, i.e. nobody can make money by having a better abstract math computation system (see DLMF and NumFOCUS list). Following is the list of current market leaders and their open-source competitors/clones:

However, closed-source projects tend to die in a decade or two (see the comments by Tim Daly). For up-to-date information, check out ISSAC (SIGSAM), Computeralgebra-Rundbrief, ICMS, JSS, and swMATH.

I must point out that RStudio is the only open-source alternative that has been able to overtake the proprietary alternatives, mainly because statistical computations can help you make money. Moreover, though the Anaconda distribution has become the standard open-source alternative to all proprietary computational softwares, there still isn’t any Python library for “structural computations” (Galois representations, isomorphisms, counting points on varieties, etc.). I believe that this is because SageMath’s motto “building the car instead of reinventing the wheel” turned it into bloatware since they integrated other well-developed open source programs like R, FriCAS, Maxima, and SymPy using Python (the glue language of scientific computing), instead of just building the Python libraries (like CyPari2) which would act as an alternative to Magma, fulfilling the original motto “Software for Arithmetic Geometry Experimentation”. To overcome this failure, SageMath evolved into a cloud-based subscription service called CoCalc to generate revenue for funding the massive project. However, the main contributions seem to come from the developments related to the LMFDB project. It might also be useful to look at the ANTS proceedings.

Mix and match

Open-source softwares has been a lifesaver for students in countries like India, where our universities couldn’t afford the proprietary softwares. I learned numerical analysis on GNU Octave and number theory on SageMath. Note that, there exist computer algebra systems that follow the philosophy of accepting a given general-purpose programming language and extending it by a set of algebraic capabilities:

That is, people have tried achieving a full-fledged CAS in many other programming languages, but all of them are half-baked due to the lack of contributors. In fact, using GiNaC we can get a symbolic extension for GNU Octave (nowadays can also use SymPy) and its fork Pynac provides the backend for symbolic expressions in SageMath (before that Maxima was used). Moreover, SymEngine is planned to be used as an optional fast symbolic core for SymPy since it is much faster than Pynac and a bit faster than GiNaC. However, for doing computational experiments in higher arithmetic we will have to learn multiple languages:

Any mathematician who is serious about doing extensive computational work in algebraic number theory and arithmetic geometry is strongly urged to become familiar with all three systems [Sage, Pari, and Magma] since they all have their pros and cons. Pari is sleek and small, Magma has much unique functionality for computations in arithmetic geometry, and Sage has a wide range of functionality in most areas of mathematics, a large developer community, and much unique new code. – SageMath October 2008 Bordeaux meeting

There is similar advice from Andrew Sutherland and John Voight in this blog post by Frank Calegari.

Python: General-purpose programming language (open-source)

During my undergraduate studies, I had the perception that C++ is “the langauge” for scientific computations (see the table above). In fact, the C/C++ libraries like GMP, MPFR, and GNU MPC serve as the foundations for various other higher arithmetic computation libraries. However, with the advent of Cython (not to be confused with CPython, which is the original Python implementation), Python has become “the new langauge” for scientific computations. The key advantage of Python over C++ is that Python provides better code readability than C++ (like using whitespaces instead of curly braces). In fact, all these C/C++ libraries can now be accessed via SageMath using Python-based syntax. For example, in 2016, Robert J. Lemke Oliver used SageMath and C++ to verify the observations by Kannan Soundarajan, leading to the famous discovery of biases in the distribution of consecutive primes.

To begin the Python journey, we will need the following packages:

*Note that Fedora will automatically install the required dependecies like mpmath, Cython, matplotlib etc.

**Requires the actual PARI/GP package and the PARI library development package.

If you want GUI version of Python development environment then can get IDLE by installing the package python3-idle. We will use Vim as the text editor for writing Python scripts. Note that we don’t need to add a file extension, however for proper syntax highlighting we will use the file extension .py. Therefore, it would be helpful to install the python-mode plugin (instructions). Following are some the key mappings for some of the python-mode commands (documentation):

Note that the text-wrap is disabled by default, with the maximum line length of 79 characters. Moreover, it uses pylint to check code at every save. I couldn’t figureout how to use code completion via rope.

One can learn the basics by going through Hands-on Python Tutorial by Andrew N. Harrington or skimming through the tutorial available in the official documentation. If you prefer video lectures, then the MIT OCW Introduction to Computer Science and Programming in Python is a good option (many other free tutorials are also there on YouTube).

Python is a multi-paradigm, call by object, statically scoped, both dynamically and strongly typed programming language.

Therefore, though Python is an object-oriented language, it doesn’t imply that it doesn’t support other programming paradigms (wikipedia). Also, it would be wise to keep in mind the floating point arithmetic limitations that these general purpose programming languages. For example, if using Python as a calculator, we will get “wrong” answers when using decimals:

# Python3 examples
>>> 3.3 - 1.1  # Do not depend on the exactness of floating point arithmetic, even for apparently simple expressions! 
2.1999999999999997
>>> 0.1+0.2  # so here 0.3 is not equal to 0.1 + 0.2
0.30000000000000004
>>> format(.1, '.20f')  # 0.1 does not has exact representation in binary floating point
'0.10000000000000000555'

To familiarize yourself with the useful maths libraries in Python (for example, mpmath provides support for arbitrary-precision floating point arithmetic, one can use the notes for the Maths with Python course at the University of Southampton or Fundamentals of Computer Programming course by Clinton Bradford at Purdue University. If you prefer video lectures, then the MIT OCW Introduction to Computational Thinking and Data Science is a good option. For example, we can get “correct” answers for float arithmetic by using decimal module:

# trying to bypass float arithmetic problem
>>> from decimal import *
>>> Decimal('3.3') - Decimal('1.1')
Decimal('2.2')
>>> Decimal('0.1') + Decimal('0.2')
Decimal('0.3')
>>> Decimal('0.1') == 0.1
False
>>> Decimal('3.5') == 3.5
True

Examples

For elementary numer theory examples, you can use either the course notes for this (very) short course in Number Theory by Robert Campbell at the University of Maryland, Baltimore County (course website) or the the companion website for the book “An Illustrated Theory of Numbers” by Martin H. Weissman. However, here we will take some examples from SageMath and solve them using Python libraries.

• Plotting elliptic curves over affine plane

  • SageMath script and output

     p=plot(EllipticCurve([0,0,0,3,0]), gridlines='true', color=hue(0.7),xmin=-4, xmax=4, ymin=-3, ymax=3, legend_label='$y^2=x^3+3x$')
     p+=plot(EllipticCurve([0,0,0,2,0]), gridlines='true', color='cornflowerblue',xmin=-4, xmax=4, ymin=-3, ymax=3, legend_label='$y^2=x^3+2x$')
     p+=plot(EllipticCurve([0,0,0,1,0]), gridlines='true', color='red',xmin=-4, xmax=4, ymin=-3, ymax=3, legend_label='$y^2=x^3+x$')
     p+=plot(EllipticCurve([0,0,0,-1,0]), gridlines='true', color='orange',xmin=-4, xmax=4, ymin=-3, ymax=3)
     p+=plot(EllipticCurve([0,0,0,-2,0]), gridlines='true', color=hue(0.2),xmin=-4, xmax=4, ymin=-3, ymax=3, legend_label='$y^2=x^3-2x$')
     p+=plot(EllipticCurve([0,0,0,-3,0]), gridlines='true', color=hue(0.3),xmin=-4, xmax=4, ymin=-3, ymax=3, legend_label='$y^2=x^3-3x$')
     p.set_legend_options(shadow=False, loc=2)
     show(p)
     p.axes_labels(['$x$','$y$'])
     p.save('ec.pdf')
    

    

  • Python script and output

  to-do

• Plotting elliptic curve modulo prime

  • SageMath script and output

    E=EllipticCurve(GF(1021),[0,-1,-1,0,0])
    A=E.plot()
    show(A)
    A.save('v1.pdf')
  

  

  • Python script and output

  to-do

• Comparing the prime counting functions

  • SageMath script and output

   P = plot(Li(x), (x,2,10000), linestyle="--", thickness=2, rgbcolor=(0,0,0), legend_label='$\\mathrm{Li}(x)$')
   Q = plot(prime_pi(x), (x,2,10000),thickness=2, rgbcolor=(0,0,0), legend_label='$\\pi(x)$')
   R = plot(x/log(x), (x,2,10000), linestyle=":", thickness=3, rgbcolor=(0,0,0), legend_label='$x/\\log(x)$')
   (P+Q+R)
  

  

  • Python script and output

  to-do

• Plotting Riemann zeta function for real inputs (trivial zeros)

  • SageMath script and output

    p=plot(zeta(x), (x,-13,-1), legend_label='$\\zeta(x)$')
    show(p)
  

  

  • Python script and output

  to-do

• Plotting the real and imaginary parts of the Riemann zeta function $\zeta(1/2 + it)$ for $0 < t < 30$

  • SageMath script and output

    i = CDF.gen()
    v = [zeta(0.5 + n/10 * i) for n in range(300)]
    L = [(z.real(), z.imag()) for z in v]
    line(L)
  

  

• Comparing the argument and absolute values of Riemann zeta function $\zeta(1/2 + it)$ for $0 \leq t \leq 50$

  • SageMath script and output

    i = CDF.0
    p1 = plot(lambda t:  arg(zeta(0.5+t*i)), 1, 50,linestyle="--", rgbcolor=(0,0,0), legend_label='$\\mathrm{arg}(\\zeta(0.5+it))$')
    p2 = plot(lambda t:  abs(zeta(0.5+t*i)), 1, 50, rgbcolor=(0,0,0), legend_label='$|\\zeta(0.5+it)|$')
    p1+p2
  

  

• Plotting complex valued functions

  • SageMath script and output

    complex_plot(x^4-1,(-5,5),(-5,5))
  

  

• Ideal factorization

  • SageMath script and output

    K.<a>=NumberField(x^3-19)
    I = K.ideal(3)
    F = I.factor()
    F
  

    (Fractional ideal (3, 1/3*a^2 + 1/3*a + 1/3))^2 * (Fractional ideal (3, 1/3*a^2 + 1/3*a - 2/3))
  

  • Python script and output

  to-do

  to-do

Magma: Domain-specific programming language (proprietary)

There exist individual C/C++ libraries, like FLINT and m4ri, which are much more efficient than Magma. In fact, Magma itself uses some of the C/C++ libraries like ATLAS, GMP, GMP-ECM, MPC, and MPFR. However, there aren’t enough libraries available to enable SageMath completely replace Magma.

In the USA, because of the Simons Foundation Agreement, you can get access to Magma for free by contacting your department’s IT support staff. You should be able to access its full-version on your department’s computer clusters and student-version on your personal computer (installation steps). Note that the student-version is available only for the outdated 32-bit architecture, therefore you might also need to install additonal packages like glibc.i686 in Fedora (details) or ia32-libs-multiarch in Ubuntu (details).

We will use Vim as the text editor for writing Magma scripts. Note that, just like for Python scripts, we don’t need to add any specific filename extensions to the text files in order to be able to “load them” in Magma (details). However, one can use filename extension .m to get Objective-C syntax highlighting (objc.vim) and to be consistent with the extension used for magma package files (details). Moreover, we can run the code without having to leave the text editor by adding the following to .vimrc file (source):

map <F2> :w<CR>:!magma %<CR> "once you press <F2> in normal mode, it first saves your file and then run the file with magma.
imap <F2> <Esc>:w<CR>:!magma %<CR> "once you press <F2> in insert mode, it first leaves insert mode, then saves the file and then run the file with magma.

One can learn the basics by going through the “First Steps in Magma” or skimming through the first chapter of the “Handbook of Magma Functions”, you will be ready to start using Magma for solving math problems. Note that Magma supports the old-school procedural programming paradigm with an ability to implement functional style.

Magma is an imperative, call by value, statically scoped, dynamically typed programming language, with an essentially functional subset.

Examples

For some elementary numer theory examples, see the Micro introduction into Magma available on the legacy website of Harvard University’s Department of Mathematics. However, we will take some examples from SageMath and solve them using Magma. Note that, unlike SageMath, Magma doesn’t have access to any plotting library like matplotlib. Therefore, we can’t reproduce the graphical examples we saw above.

• Ideal factorization

  • SageMath script and output (doc)

    K.<a>=NumberField(x^3-19)
    I = K.ideal(3)
    F = I.factor()
    F
  

    (Fractional ideal (3, 1/3*a^2 + 1/3*a + 1/3))^2 * (Fractional ideal (3, 1/3*a^2 + 1/3*a - 2/3))
  

  • Magma script and output

    R<x>:=PolynomialRing(Integers());
    f:=x^3 - 19;
    K<y>:=NumberField(f);
    O:=MaximalOrder(K);
    I:=3*O;
    Factorization(I);
  

    [
    <Prime Ideal of O
    Two element generators:
        [3, 0, 0]
        [2, 0, 2], 1>,
    <Prime Ideal of O
    Two element generators:
        [3, 0, 0]
        [0, 2, 1], 2>
    ]
  

Conclusion

William Stein has played a key role in starting this open-source mathematical software revolution. For example, there exist free online textbooks which use SageMath to teach elementary number theory, linear algebra, abstract algebra, graph theory, and game theory. However, it would have been great if the academic community could understand the value of open-source Python-based Magma.

Fortunately, the OSCAR project, supported by the German Research Foundation (DFG) since January 2017, has the potential of becoming an open-source Julia-based Magma. I believe that unlike SageMath, OSCAR will be able to replace Magma, since like Magma is maintained by the Computational Algebra Group at the University of Sydney, OSCAR is maintained by the Algebra, Geometry and Computer Algebra group at TU Kaiserslautern (usecase example). Moreover, Julia has simple syntax like Python (Python equivalent is CPython: interpreted language for fast software development), is as fast as C++ (Python equivalent is Cython: compiled language for fast computations) and uses a LLVM just-in-time (JIT) compiler similar to the JVM JIT compiler of Java (Python equivalent is PyPy, similar to LuaJIT: dynamic language for fast computations).

In my opinion, the research papers funded by public taxes should be freely available (like free beer) and the softwares needed to do tax-funded research should also be freely available (like free speech). Recently, there has been a surge in researchers using SageMath or Python for doing computations in higher arithmetic. For example, this arithmetic geometry paper uses Python for computations.

\usepackage{graphicx} %add images
\graphicspath{ {./figures/} } %images are kept in a folder under the directory of the main document.
\usepackage{subcaption} %to add multiple subfigures.

However, for better integration into the pdf document we should be using vector graphics and it can be done with the help of following programs/packages:

Graphs

pgfplots

For including 2D and 3D plots, we can use the pgfplots package. Read the documentation. First include the following in the preamble:

\usepackage{pgfplots} %draws function plots using pgf/tikz
\pgfplotsset{compat=1.16} %running latest version of pgfplots
\usepgfplotslibrary{external} %avoid recompiling unchanged graphs
\tikzexternalize[prefix=./figures/] %images will be stored in a folder under the working directory to avoid recompling unchanged files

For example, we can plot 3D surfaces by following methods:

\begin{tikzpicture} %https://tex.stackexchange.com/a/359914/
\begin{axis}[title=$f^{-1}(-1)$: Hyperboloid of 1 sheet,axis equal]
\addplot3[surf,domain=0:360,y domain=-2:2] ({cosh(y)*cos(x)},{cosh(y)*sin(x)},{sinh(y)});
\end{axis}
\end{tikzpicture}

  \begin{tikzpicture}%https://tex.stackexchange.com/a/28775/
  \begin{axis}[title=$f^{-1}(0)$: Double cone, domain=0:5, y domain=0:2*pi,xmin=-10, xmax=10, ymin=-10, ymax=10, samples=20]
  \addplot3 [surf,z buffer=sort] ({x*cos(deg(y))}, {x*sin(deg(y))}, {x});
  \addplot3 [surf,z buffer=sort] ({x*cos(deg(y))}, {x*sin(deg(y))}, {-x});
  \end{axis}
  \end{tikzpicture}

The downside of this method is that it will increase the compliation time if prefix option is not used (since pdfLaTeX is limited to single CPU thread). We can also use contour gnuplot and \addplot gnuplot to extend the built-in capabilities of pgfplots by means of gnuplot’s math library, although their use is optional.

Matplotlib

Another way of including 2D and 3D plots is to use the Python library Matplotlib. Read this guide. I will illustrate the steps involved by the following plotting the 2D vector field $X=-y\frac{\partial}{\partial x} + x\frac{\partial}{\partial y} and its flow lines:

1. Save and run the following Python scripts in the “figures” subfolder inside the folder containg the main LaTeX file (source1, source2):

import numpy as np
import matplotlib.pyplot as plt

#use quiver plot to plot the vector field
x,y = np.meshgrid(np.linspace(-5,5,10),np.linspace(-5,5,10))
u = -y
v = x
plt.quiver(x,y,u,v)

#save as pdf and pgf
plt.savefig('field.pdf')
plt.savefig('field.pgf')

import numpy as np
import matplotlib.pyplot as plt

#use stream plot to create the flow lines
x,y = np.meshgrid(np.linspace(-5,5,10),np.linspace(-5,5,10))
u = -y
v = x
plt.streamplot(x,y,u,v)

#save as pdf and pgf
plt.savefig('flow.pdf')
plt.savefig('flow.pgf')

1. Add the following lines in the preamble of the main LaTeX file:

\usepackage{pgf}
\usepackage{import} %import images from different folder 

1. Finally, to include pgf version of these plots in the document, use:

\resizebox{0.75\textwidth}{!}{\import{./figures/}{field.pgf}}

\resizebox{0.75\textwidth}{!}{\import{./figures/}{flow.pgf}}

You can also use figure or center environment as per your needs.

1. We will get the following plots with proper font and resolution:

Alternatively, you can insert the pdf file by following this guide.

Diagrams

PGF/TikZ

Manually

Apart from all these tools, one can directly use TikZ package to manually draw things like flowcharts. Read the documentation for details. For example, we can add the following to the preamble:

\usepackage{tikz}
\usetikzlibrary{shapes,arrows, chains, matrix, calc, trees, positioning, fit}
\usetikzlibrary{external}
\tikzexternalize[prefix=./figures/] %images will be stored in a folder under the working directory

Then we can draw function mapping:

\begin{tikzpicture}[line width=1pt,>=latex]
			\node (a1) {$\mathbb{Q}_p(\sqrt{u}) \ \bullet$};
			\node[below=of a1] (a2) {$\mathbb{Q}_p(\sqrt{-p})\ \bullet$} ;
			\node[below=of a2] (a3) {$\mathbb{Q}_p(\sqrt{-p\cdot u})\ \bullet$};

			\node[right=4cm of a1] (b1) {$ \bullet \ \mu_p$};
			\node[right=4cm of a2] (b2) {$ \bullet \ \lambda_p$};
			\node[right=4cm of a3] (b3) {$ \bullet \ \mu_p\lambda_p$};

			\node[shape=ellipse, draw, minimum size=3cm,fit={(a1) (a3)}] {};
			\node[shape=ellipse, draw, minimum size=3cm,fit={(b1) (b3)}] {};

			\node[below=1cm of a3] {$\mathcal{E}$};
			\node[below=1cm of b3] {$\mathcal{Q}$};

			\draw[->] (a1) -- (b1);
			\draw[->] (a2) -- (b2);
			\draw[->] (a3) -- (b3);
\end{tikzpicture}

and flowcharts

\begin{tikzpicture}[>=latex']
\node[left] at (0,0) (input) {$p$};
\node[block] at (4,0) (block) [text=white]{\textbf{Reciprocity Law}};
\node at (9,0) (output) {$\chi_\tau(\Frob_p)$};
\draw[very thick, ->] (input) -- (block);
\draw[very thick, ->] (block) -- (output);
\end{tikzpicture}

Using GeoGebra Classic

The easiest way to include simple diagrams is to use GeoGebra, since it gives us the option of exporting the Graphics View as PGF/TikZ code. Installing GeoGebra Classic 6 is a dependency hell (ex1 and ex2). Therefore, we will just use the online version: https://www.geogebra.org/classic For example, add the following to the preamble:

\usepackage{pgf,tikz}
\usetikzlibrary{arrows}%you can see what extra packages are needed by going through the exported tex file

Then draw the following diagram in GeoGebra

And add the following corresponding code in the tex file:

\begin{tikzpicture}[line cap=round,line join=round,>=triangle 45,x=2.1cm,y=2.1cm]
\clip(1.5,-0.5) rectangle (10,3);
\draw (2,0)-- (4.83,0);
\draw (2,0)-- (3.41,1.41);
\draw (3.41,1.41)-- (4.83,0);
\draw (2,0.91) node[anchor=north west] {\scriptsize{$\overline{CA} \, = \, 2$}};
\draw (4.15,0.92) node[anchor=north west] {\scriptsize{$\overline{CB} \, = \, 2$}};
\draw (3.18,0) node[anchor=north west] {\scriptsize{$\overline{AB} \, = \, 2.83$}};
\draw (6,0)-- (6,1);
\draw (6,1)-- (7,1);
\draw (7,1)-- (7,0);
\draw (7,0)-- (6,0);
\draw (5.35,0.66) node[anchor=north west] {\scriptsize{$\overline{DE} \, = \, 1$}};
\draw (6.25,1.25) node[anchor=north west] {\scriptsize{$\overline{EF} \, = \, 1$}};
\draw (7.04,0.66) node[anchor=north west] {\scriptsize{$\overline{FG} \, = \, 1$}};
\draw (6.34,0) node[anchor=north west] {\scriptsize{$\overline{GD} \, = \, 1$}};
\draw (4.5,1.5)-- (5.5,1.5);
\draw (5.5,1.5)-- (5.5,2.5);
\draw (5.5,2.5)-- (4.5,1.5);
\draw (4.86,1.52) node[anchor=north west] {\scriptsize{$\overline{HI} \, = \, 1$}};
\draw (5.54,2.19) node[anchor=north west] {\scriptsize{$\overline{IJ} \, = \, 1$}};
\draw (4.25,2.2) node[anchor=north west] {\scriptsize{$\overline{JH} \, = \, 1.41$}};
\draw (7.5,1.5)-- (8.5,1.5);
\draw (8.5,1.5)-- (9.5,2.5);
\draw (9.5,2.5)-- (8.5,2.5);
\draw (8.5,2.5)-- (7.5,1.5);
\draw (8.7,2.73) node[anchor=north west] {\scriptsize{$\overline{NM} \, = \, 1$}};
\draw (9,2.08) node[anchor=north west] {\scriptsize{$\overline{ML} \, = \, 1.41$}};
\draw (7.83,1.5) node[anchor=north west] {\scriptsize{$\overline{LK} \, = \, 1$}};
\draw (7.2,2.22) node[anchor=north west] {\scriptsize{$\overline{KN} \, = \, 1.41$}};
\draw (1.94,2.51)-- (2.94,1.51);
\draw (2.94,1.51)-- (3.94,2.51);
\draw (3.94,2.51)-- (1.94,2.51);
\draw (2.8,2.76) node[anchor=north west] {\scriptsize{$\overline{OQ} \, = \, 2$}};
\draw (3.3,2) node[anchor=north west] {\scriptsize{$\overline{QP} \, = \, 1.41$}};
\draw (1.75,2) node[anchor=north west] {\scriptsize{$\overline{PO} \, = \, 1.41$}};
\begin{scriptsize}
\fill [color=black] (2,0) circle (1.5pt);
\draw[color=black] (1.9,-0.08) node {$A$};
\fill [color=black] (4.83,0) circle (1.5pt);
\draw[color=black] (4.88,-0.08) node {$B$};
\fill [color=black] (3.41,1.41) circle (1.5pt);
\draw[color=black] (3.46,1.5) node {$C$};
\fill [color=black] (6,0) circle (1.5pt);
\draw[color=black] (6.05,-0.08) node {$D$};
\fill [color=black] (6,1) circle (1.5pt);
\draw[color=black] (6.05,1.09) node {$E$};
\fill [color=black] (7,1) circle (1.5pt);
\draw[color=black] (7.04,1.09) node {$F$};
\fill [color=black] (7,0) circle (1.5pt);
\draw[color=black] (7.05,-0.08) node {$G$};
\fill [color=black] (4.5,1.5) circle (1.5pt);
\draw[color=black] (4.4,1.4) node {$H$};
\fill [color=black] (5.5,1.5) circle (1.5pt);
\draw[color=black] (5.63,1.4) node {$I$};
\fill [color=black] (5.5,2.5) circle (1.5pt);
\draw[color=black] (5.53,2.6) node {$J$};
\fill [color=black] (7.5,1.5) circle (1.5pt);
\draw[color=black] (7.45,1.4) node {$K$};
\fill [color=black] (8.5,1.5) circle (1.5pt);
\draw[color=black] (8.65,1.4) node {$L$};
\fill [color=black] (9.5,2.5) circle (1.5pt);
\draw[color=black] (9.6,2.58) node {$M$};
\fill [color=black] (8.5,2.5) circle (1.5pt);
\draw[color=black] (8.45,2.58) node {$N$};
\fill [color=black] (1.99,2.51) circle (1.5pt);
\draw[color=black] (1.99,2.59) node {$O$};
\fill [color=black] (2.94,1.51) circle (1.5pt);
\draw[color=black] (2.99,1.45) node {$P$};
\fill [color=black] (3.94,2.51) circle (1.5pt);
\draw[color=black] (3.99,2.59) node {$Q$};
\fill [color=black] (2.94,2.51) circle (1.5pt);
\fill [color=black] (2.71,0.71) circle (1.5pt);
\fill [color=black] (4.12,0.71) circle (1.5pt);
\fill [color=black] (3.41,0) circle (1.5pt);
\end{scriptsize}
\end{tikzpicture}

Another example, where this approach makes the job easy is the following:

Just add this code generated by GeoGebra:

\begin{tikzpicture}[line cap=round,line join=round,>=triangle 45,x=0.75cm,y=0.75cm]
\clip(2.5,-5.5) rectangle (13.5,5.5);
\draw [shift={(8,0)},fill=black,fill opacity=0.1] (0,0) -- (0:1.2) arc (0:289.87:1.2) -- cycle;
\draw [shift={(8,0)},fill=black,fill opacity=0.1] (0,0) -- (0:0.7) arc (0:197.97:0.7) -- cycle;
\draw(8,0) circle (3.75cm);
\draw(6,0) circle (1.95cm);
\draw (8,0)-- (3.24,-1.54);
\draw (3.24,-1.54)-- (9.7,-4.7);
\draw (9.7,-4.7)-- (8,0);
\draw [dash pattern=on 3pt off 3pt] (6,0)-- (4.93,-2.37);
\draw [dash pattern=on 3pt off 3pt] (6,0)-- (13,0);
\draw [shift={(8,0)},->] (0:1.2) arc (0:289.87:1.2);
\draw [shift={(8,0)},->] (0:0.7) arc (0:197.97:0.7);
\begin{scriptsize}
\fill [color=black] (8,0) circle (1.5pt);
\draw[color=black] (7.8,-0.2) node {$O$};
\fill [color=black] (13,0) circle (1.5pt);
\draw[color=black] (13.18,-0.12) node {$B$};
\fill [color=black] (6,0) circle (1.5pt);
\draw[color=black] (5.98,0.25) node {$O'$};
\fill [color=black] (8.6,-0.08) circle (1.5pt);
\draw[color=black] (8.74,-0.22) node {$D$};
\fill [color=black] (3.24,-1.54) circle (1.5pt);
\draw[color=black] (2.94,-1.4) node {$E'$};
\fill [color=black] (9.7,-4.7) circle (1.5pt);
\draw[color=black] (9.84,-4.44) node {$E_1'$};
\fill [color=black] (4.93,-2.37) circle (1.5pt);
\draw[color=black] (4.82,-2.08) node {$G'$};
\draw[color=black] (7.5,0.8) node {$2\theta_1'$};
\draw[color=black] (8.1,0.4) node {$2\theta_0'$};
\end{scriptsize}
\end{tikzpicture}

tikz-cd

Manually

We can draw commutative diagrams using tikz-cd package. Read its documentation for details. Firstly you will have to add the following to the preamble:

\usepackage{tikz-cd}

%%homebrewcommands

\newcommand{\delbar}{\overline{\partial}}
\newcommand{\Om}{\Omega}
\newcommand{\mz}{\mathcal{Z}}

\DeclareMathOperator{\ceco}{\check{H}}

Then add the following code:

\[\begin{tikzcd}[column sep = 1.5em]
0\arrow{r} & \mz^{p,\ell}(M) \arrow[hookrightarrow]{r} & \Om^{p,\ell}(M) \arrow{r}{\delbar} & \mz^{p,\ell+1}(M) \arrow{r}{\Delta} & \ceco^1(M,\mz^{p,\ell}) \arrow{r} & 0 \arrow{r} & \ceco^1(M,\mz^{p,\ell+1}) \arrow{d}{\Delta}\\
& \cdots & 0\arrow{l} & \ceco^3(M,\mz^{p,\ell}) \arrow{l}& \ceco^2(M,\mz^{p,\ell+1})\arrow{l}[swap]{\Delta} & 0 \arrow{l} & \ceco^2(M,\mz^{p,\ell})\arrow{l}
\end{tikzcd}\]

to get

Using quiver app

The easiest way to include commutative diagrams is by using quiver app available online at https://q.uiver.app/ For example, add the following to the preamble:

\usepackage{tikz-cd}

Then draw the following diagram using quiver app

And add the corresponding exported code in the tex file:

\[\begin{tikzcd}
	&& {Z} \\
	\\
	&& {X\times_S Y} \\
	\\
	{X} &&&& {Y} \\
	\\
	&& {S}
	\arrow[from=1-3, to=5-1]
	\arrow[from=1-3, to=5-5]
	\arrow["{f}"', from=5-1, to=7-3]
	\arrow["{g}", from=5-5, to=7-3]
	\arrow[from=3-3, to=7-3]
	\arrow["{\exists !}", from=1-3, to=3-3, dashed]
	\arrow["{\pi_X}", from=3-3, to=5-1]
	\arrow["{\pi_Y}"', from=3-3, to=5-5]
\end{tikzcd}\]

Inkscape

The easiest way to include diagrams as vector graphics is by using Inkscape as follows:

1. Draw the desired diagram in Inkscape, and enclose mathematical symbols in $...$. If you don’t know how to use Inkscape then just go to Help > Tutorials > Inkscape: Basic and you will be ready to use.
2. Create a folder called “pictures” inside the folder containg the main tex file and save the diagram as svg so that you can edit it in the future if needed. Then also save PDF+LaTeX output to the same “pictures” folder: File > Save As... > Select PDF from the drop-down menu > Click Save > Choose the following options

You can also use command-line to achieve the same result:

inkscape mySVGinputFile.svg --export-area-drawing --batch-process --export-type=pdf --export-filename=output.pdf

1. Add the following code to the preamble of your main tex file (source1 and source2):

\usepackage{import} % it will enable us to access images without keeping them in the document's directory
\usepackage{calc} % enable usage of \svgscale
\usepackage{graphicx} % for inserting images
\usepackage{xcolor} % for adding colors
\usepackage{transparent} % enables usage of separate color stack for transparency

1. Now we can add “image.svg” in one of the following ways (source1 and source2):

\begin{figure}[h]
\centering
\resizebox{75mm}{!}{\import{./figures/}{image.pdf_tex}}
\caption{Your figure}
\label{figure:example}
\end{figure}


\begin{figure}[h]
 \centering
\def\svgwidth{0.75\columnwidth}                 %Using \def\svgwidth{desired width} instead of \resizebox will preserve the font size
\import{./figures/}{image.pdf_tex}
\caption{Your figure}
\label{figure:example}
\end{figure}


\begin{figure}[h]
\centering
\def\svgscale{0.75}
\import{./figures/}{image.pdf_tex}
\caption{Your figure}
\label{figure:example}
\end{figure}

You can also write scripts to make this process easier. The main advantage of Inkscape is that there’s hardly any command or action that is impossible to do from keyboard. Linux users may not get the expected results with the key combinations starting with Alt key if the Window Manager catches those key events before they reach the inkscape application. One solution would be to change the WM’s configuration accordingly.

Preparation

The plan is to use Vim as the text-editor and MuPDF as the pdf-viewer since both can be accessed solely via keyboard, while keeping all the desired features from other LaTex editor like “live pdf preview” and “compile using latexmk” from GTK based Gummi and “forward and backward search to switch between the sources and the PDF” and “detailed compilation errors” from Qt based TexMaker.

We will need the following packages to begin with:

If you want GUI version of Vim then can get gVim by installing the package vim-X11. For gVim you will have to modify these steps accordingly.

Vim configuration

Once you finish learning basics using vimtutor, you are ready to start configuring Vim as per your requirements.

.vimrc file creation

To start Vim with all the favorite option settings and mappings, one writes them in what is called the vimrc file. The vimrc file can contain all the commands that you type after a colon. Vim executes the commands in this file when it starts up. Read the documentation for more details. Note that Vim 8.0 onwards, if Vim is started normally and no user vimrc file is found, the defaults.vim script is loaded. This will set ‘compatible’ off, switch on syntax highlighting and a few more things. See this reddit post for a discussion about this. To disable loading of defaults.vim completely add let skip_defaults_vim=1 to /etc/vimrc (see this).

Since vimrc is something personal that evolves over times as per one’s usage requirements, it’s not recommended to blindly copy it from the internet. I used this example as reference for creating the following ~/.vimrc

set nocompatible               " get rid of Vi compatibility mode. 
filetype plugin indent on      " filetype detection[ON] plugin[ON] auto-indent[ON] depending on filetype
set t_Co=256                   " enable 256-color mode.
syntax enable                  " enable syntax highlighting (can also use syntax on).
colorscheme morning            " I am using it with solarized (dark) color pallet in xfce4-terminal.
set hidden                     " allows you to re-use the same window without saving it first and keep an undo history for all the files using the same window.
set number                     " show line numbers to the left
set ruler                      " Always show cursor position.
set laststatus=2               " always display the status line so that you can see the current mode, file name, file status, ruler, etc. 
filetype indent on             " activates indenting for files
set hlsearch                   " highlight searched phrases.
set ignorecase                 " Make searches case-insensitive.
set smartcase                  " Make search case-sesitive when using capital letters
set autoindent                 " When opening a new line and no filetype-specific indenting is enabled, keep " the same indent as the line you're currently on.
set backspace=indent,eol,start " Allow backspacing over autoindent, line breaks and start of insert action
set tabstop=8                  " sets tab stops to 8 characters wide
set softtabstop=4              " makes the Tab key intent by 4 spaces
set shiftwidth=4               " width of autointents set to 4 spaces
set expandtab                  " converts tabs to white space
set mouse=a                    " Enable use of the mouse for all modes
set wrap linebreak nolist      " Soft wrapping text. To move the cursor up and down within wrapped line use the commands gk and gj.

You can use this guide to create a backup of your vimrc file using GitHub so that you can easily configure Vim on any other computer (how to find git folder). You can follow the official guide to set-up GitHub for this purpose.

vimtex plugin installation and configuration

You can install vimtex manually or using a plugin manager (see this guide). We will use the vim-plug plugin manager to install vimtex. Firstly, we will download plug.vim and put it in the ~/.vim/autoload directory using following code:

curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

Next we will enable vimtex by adding the following section to the ~/.vimrc file:

call plug#begin('~/.vim/plugged')  " Specify the directory ~/.vim/plugged for plugins. Avoid using standard Vim directory names like ~/.vim/plugin
Plug 'lervag/vimtex'               " Shorthand notation for fetching the vimtex plugin from https://github.com/lervag/vimtex
call plug#end()                    " To update &runtimepath and initialize plugin system

Once the vimrc is properly configured, restart Vim or reload the vimrc file (make sure that git is installed for this to work). Finally, run the :PlugInstall command to start the installation of vimtex plugin. Vim-plug will download all the packages directly from GitHub and put them into the ~/.vim/plugged directory and load them whenever Vim is loaded. To update plugins use :PlugUpdate command and to remove them use :PlugClean command. Also, to update vim-plug itself, use :PlugUpgrade command.

Now we will set it up following the official documentation of vimtex, and adding the following to ~/.vimrc:

let g:tex_flavor = 'latex'           " Vim ships with some support for plain TeX, ConTeXt, and LaTeX files. This means that the `.tex` extension is ambiguous. Vimtex is only activated for LaTeX files with 'filetype' set to `tex`.
let g:vimtex_view_method = 'mupdf'   "  Set the pdf viewer. MuPDF supports forward and backward search via xdotool. For backward search use :VimtexRSearch command. Forward search will only take you to the correct page.  Backward search will take you to the line in Vim that corresponds to the first line of the current page in MuPDF.
let g:vimtex_compiler_latexmk= {'options' : ['-pdf', '-shell-escape', '-verbose', '-file-line-error', '-synctex=1', '-interaction=nonstopmode',],} " we need to enable -shell-escape to be able to use externalization library for avioiding recompiling unchanged diagrams/graphs created using tikz/pgfplots

Note that by default the following desired options are already there:

• latexmk is the compiler which does compilation as soon as you save the file (continous compilation, better than live preview)
• auto-completion is enabled,
• BibTex is used for bib files,
• fold types are enabled,
• intentation in both tex and bib are enabled,
• vimtex will open the pdf viewer automatically after compilation
• forward search is enabled, i.e. it will perform a forward search to the current cursor position when the first invocation of the pdf viewer happens. It uses SyncTex and requires xdotool to work with MuPDF (also prevents opening multiple instances of pdf).

You can further customize by adding snippets as demonstrated in various blog posts (ex1, ex2, ex3 and ex4).

Usage instructions

Following are the useful key mappings for the various vimtex commands:

We have many other shorthand keymaps like:

Note that vimtex supports most multi-file documents. The main method uses a recursive search algorithm that should find the main LaTeX file in most cases. Read the documentation for more details.

Update (Aug 30, 2023): I have moved to Windows 11 with WSL. We have Microsoft Windows supporters like SUSE (openSUSE), Ubuntu (Debian Sid), and RedHat/Oracle Linux (Fedora) for WSL, Azure,… and competitors like Apple macOS (UNIX) and Google chromeOS (ChromiumOS).

Open-source software is better

Open-source softwares are great because they have scope for technical superiority by allowing transparency. In fact, this is good enough incentive for many big corporations like Intel (server processors), IBM (Red Hat Enterprise Linux), Microsoft (Azure), Google (Chrome), Novell (SUSE Linux), etc. and smaller companies like Canonical (Ubuntu), etc. to sponsor Linux Foundation and other commmunity-driven Linux projects like GNOME (with major contributions coming from Red Hat), KDE (with major contributions coming from SUSE), and Debian (with major contributions coming from Canonical). However, there are still some popular Linux related projects like Arch Linux, Gentoo, Cinnamon DE and Xfce that are being developed by the community at a steady pace without any corporate funding. Though these community-only projects are really powerful and allow the user to create a custom workflow, they tend to attract only the ones with considerable knowledge of Linux and a desire to contribute.

For normal consumer products like desktops and laptops, Microsoft Windows has the monopoly because it comes pre-installed for free and is supported by all softwares (games) and the latest hardware. However, in the enterprise world where mainframe computers and servers need to be really secure, Linux distributions like Ubuntu, RHEL, CentOS, SUSE, and Debian tend to be more popular. Among these, Debian is the only one which is not owned by any specific company, but companies can directly fund the long-term-support. Ubuntu, which derives its source from Debian Sid, is quite popular among tech start-ups because of its freemium kind of business model where anyone can download it for free and one only needs to pay for extended service (similar to the extreme popularity of the free-to-play games like Fortnite and PUBG). In fact, it was marketed so well that it is the most popular Linux distribution among people who just wants a better OS than Windows but can’t afford to be a part of the Mac ecosystem.

Ubuntu’s major contribution has been the LightDM display manager. However, neither Ubuntu nor Debian is the market leader because they lack fast-paced innovation. RHEL has been the market leader for enterprise linux for a long time, mainly because it has invested a lot in innovation. Following is a rough representation of how it works in present:

Update: CentOS development was stopped on Dec 08, 2020 after being bought by IBM. However, the development of Rocky Linux has started.

The second most popular enterprise Linux distro is SUSE, it is the oldest commercial Linux distro. In recent years, it has also reorganized itself by starting community driven openSUSE project similar to the successful Fedora project to speed up innovation:

This new Tumbleweed was a result of the ‘merger’ of Old Tumbleweed and Factory in November 2014. Recently, AMD to complement its great technological advancements have started making significant contributions to the Linux kernel. In fact, it is also one of the primary sponsors of openSUSE project. In my opinion, openSUSE Tumbleweed is a good comeptitor for Fedora since it strives to provide access to latest packages using rolling release instead of short lived point release. On the other hand, openSUSE Leap is a good competitor for Ubuntu LTS since it gives an option for upgrading to much superior SUSE enterprise.

I especially like the open-source softwares like these, which let individual users access them for free while charging big corporations for techincal support. Some other examples are, Anaconda by Travis Oliphant (it also helps fund development of NumPy etc. via NumFOCUS) and MySQL by Oracle Corp.

Mix and Match

Now a days there are many emerging comeptitors like Manjaro and Solus competing with Ubuntu and its forks for the non-technical users. I was introduced to linux as a non-techincal user in 2006 when I ran Ubuntu in a virtual box on my Windows XP desktop PC, thanks to the free CD I got with the PC World magazine I had bought during summer vacations. I have been using Ubuntu-based distros (I replaced Windows 7 with Lubuntu on my Netbook) since Windows XP died in 2014, and it has been a nice journey. Whenever I ran into some trouble, I just copy-pasted codes from some blog/forum and ignored issues as long as work got done. For example, I chose not to use WiFi over using Windows 8 when Realtek drivers were breaking on Ubuntu (had no idea it has to do something with Linux Kernel). But now since I have built my own PC for the first time, I have a much better knowledge of the hardware and find it difficult to just shrug shoulders as long as system boots. Theorefore, Ubuntu can no more fulfill my needs and I will have to move to a little less stable distro.

The main use of my new PC is to write documents in LaTeX, hence I would really like to have easy access to the latest version of texlive and text-editor (like Vim, though you will always get some version of vim-minimal pre-installed). However, stable distros like Ubuntu LTS and openSUSE Leap tend to have outdated repositories of both of these and Ubuntu tries to mitigate this with untrusted PPAs, unstable Snaps and useless Flatpaks. Moreover, compiling them from sources can be a bit tricky (eg: installing TexLive and Vim). Hence I felt the need to migrate to a distro which can give easy access to latest version of these programs. But I don’t have enough time and knowledge to be able to maintain the stability and security of rolling-release distros like Arch, openSUSE TW and Gentoo. Therefore, since I don’t need any proprietary drivers for my PC, I decided to take a middle ground and move to the pseudo-rolling/stable-release distro Fedora (13 month life cycle, new release every 6 months) which is the upstream source of the commercial Red Hat Enterprise Linux distribution. Another reason for choosing Fedora was that its community is more knowledgebable (RedHat is one of the biggest contributer to Linux kernel) and the bugs acutally get resolved (for Ubuntu there can exist an year old bugs like the initramfs bug with temporary solution in forums). One bug that has followed me in Ubuntu and Linux Mint is the momentary blackout of screen when some pop-up menu opens in applications like Google Chrome and TexMaker Settings, however I have no hope of it getting solved since there is this similar three years old bug report in the forums.

For the desktop environment I decided to go with Xfce because most of my applications use GTK (hence not KDE or LXQt), didn’t want to waste RAM for customizability (hence not GNOME 3 or MATE which are resource heavy but still lack simple in-built customization options like desktop wallpaper slideshow, requires separate pacakges like GNOME Tweaks and GNOME Extensions) and needed stability (hence not Cinnamon which lacks proper documentation and bug resolving mechanism, eg: this an year old zombie-windows bug which requires restarting cinnamon to troubleshoot, moreover most appearance tweaks depend on themes and applets created by volunteers which are not guaranteed to be actively maintained). Cinnamon is an attempt to mimic Xfce while staying close to GNOME, however its lack of documentation ability to tweak easily is a big negative. One thing I miss in Xfce is the absence of dock (using Plank is fissible but not ideal). However, it can easily be mitigated by setting up keyboard shortcuts for the frequently used applications.

I installed Fedora Xfce using Server Netinstall instead of the official Spins since they tend to be bloated and will have to install lots of updates afterwards. Fedora comes with Ubuntu’s LightDM display manager which manages the login screen, in the case of Xfce Fedora uses the Xubuntu’s LightDM GTK+ Greeter.

Update (June 22, 2021): My university changed their wired internet connection protocol and now it requires unsername and password. Hence can’t use this Netinstall method anymore. Back to using the livecd image.

Differences between Ubuntu and Fedora

Most of my Linux experience is based on Ubuntu. Following are some differences that I experienced:

Administrator vs. User

Users have the option of setting a root password in when installing a non-GNOME Fedora, but it is not required if the user creates an initial user account and selects the option to add it to the wheel group. If root password is set, then the root account is the account for the system admin. This account is disabled in Ubuntu. In Ubuntu, one performs actions that require root privileges using sudo, while in a Fedora spin, sudo is not the default method of gaining administrative permissions. Therefore, in Fedora Xfce spin with root password, root access can only be gained with su. su will ask for the root password, not the regular user password. After logging in successfully as root, one has administrative rights until terminal is closed or logged out with exit. However, for GNOME Fedora, it’s just like Ubuntu i,e. no root password set by default and user account is handled by gnome-initial-setup. Having a root password is not useful for nontechnical users.

Package management

Fedora uses different tools for package management to Debian. Here is a quick overview of how to accomplish common tasks in Fedora: (source)

In older versions, apt-get in Ubuntu corresponded to yum in Fedora. To know if a reboot is needed after doing a dnf update use either tracer or needs-restarting plugin. Unlike Debian/ubuntu, I don’t need to remeber to use autoremove after removing some package.

Moreover, like in Ubuntu we use dpkg for installing .deb files, we can use rpm in Fedora for installing .rpm files. Following is a comparison of commands:

You can also use dnf instead of rpm, just like we can use apt instead of dpkg. You can find an outdate but more extensive list in Ubuntu Wiki.

Finding packages using GUI

Fedora’s equivalent to Debian’s Synaptic package manager is dnfdragora which uses the widget abstraction library written by SUSE. Unfortuately it’s nowhere as good as Synaptic. Moreover it is extremely RAM hungry. However, in GNOME edition of Fedora there is GNOME Software Centre which makes things smoother, but requires PC reboot for installing updates. In the RPM world opeSUSE has YaST as a worthy competitor of Synaptic (but that feels too nosey, it is much more than just a package manager).

Installing packages not in repository

The equivalent of the Ubuntu restricted and multiverse repositories, that include patented and closed-source technologies and programs, is the RPMFusion repository. free is the equivalent of universe and contains potentially patent-encumbered software like gstreamer-plugins-bad or the VLC media player, while nonfree includes non-free software like proprietary 3D graphics drivers.

These repositories can easily be enabled by typing (as root):

dnf install https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm
dnf install https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

Moreover, the equivalent of the PPAs in Ubuntu is Copr. This repository can be added with

dnf copr enable user/project.

Note that just like Ubuntu PPA and Arch AUR, there is no way to verify that a package in Copr does not contain anything malicious unless you review the source code.

Linux Security Modules

Ubuntu uses AppArmor (which was earlier developed by SUSE), whereas Fedora uses SELinux. AppAromor is said to be easier to maintain than SELinux.

Installing Fedora

Firstly download and verify the iso file. Then make a bootable pendrive using the Qt app Fedora Media Writer, unfortunately it requires KDE so will install that also. I decided to use ext4 file system, which is the current default file system for Workstation and Spins, instead of the dfault file system xfs for Server/Netinstall (there are benchmarks claiming either xfs is better or ext4 is better). I followed Fedora’s recommended partitioning scheme and created the following GUID partition table:

Note that modern Linux OS don’t require separate partitions for both /boot and /boot/efi (EFI System Partition) directories. The only requirement for suing GRUB2 boot loader is that /boot directory must be on a plain ext4 or xfs partition and /boot/efi directory must be on a plain vfat partition. If one wants to install Fedora with only two paritions / and /boot then it is recommended to use systemd-boot boot loader instead. However, since we were following Fedora’s recommended partitoning scheme we created separate /boot and /home paritions with standard ext4 format instead of leaving them with /. Also, I alloted thrice the recommended size for 20 GB for / because I plan to use Timeshift which saves snapshots in root directory itself and works only with GRUB2 (very important tool, was fixed later).

Select Xfce Fedora without any extra software programs, we will manually install the ones we want. To bypass creating root password in netinstall mode, just create a user account and mark it as administrator. After installing, the first think we will do is set-up drop-down terminal following this guide. Also consider changing first day of week to Monday by following this guide.

Free and open source softwares

Vim, PulseAudio Volume Control and dnfdragora are already installed.

Note that Firefox also lets you use streaming services like Netflix which require h264/mp3/aac and widevine. Also, you can remove “Pocket” from Firefox following these steps.

Unfortunately, Signal doesn’t have an official Fedora repo, however it can be installed using Copr.

To find more softwares, use: dnfdragora > Groups > Xfce desktop option.

Finally, if you need Windows to access applications like OneNote instead of Xournal++ then you can install it by running Win 10 virtual machine, by using virtIO drivers along with virtualization, that is choose i440FX chipset, UEFI without secure boot, QXL video and virtio for Virtual Disk and Virtual Network Interface (make sure to update all the missing Windows drivers using the virtIO iso file). Alternatively, dual-boot Fedora with Windows following this guide and remember to disable HDCP support overrride in Display settings of AMD Radeon driver if not using HDMI port.

Restricted softwares

Note that Google Chrome poorly integrates with the Linux. For example, if you change the default DPI of 96 or font scaling for the display then Google Chrome text and images get blurred and you have to fix it yourself (possible solution 1 and solution 2).

/usr/bin/google-chrome-stable --force-device-scale-factor=1 %U

It also poorly integrates with Xfce since it attempts to creat its own default keyring for password encryption, so either you will have to create another password or disable it.

DNF plugins

We will install the following two plugins:

Xfce panel plugins appearance

Few panel plugins like Datetime, Places, PulseAudio and ScreenShooter are already installed.

Note that the weather plugin has known bugs, like using inaccurate data from Meteorologisk institutt instead of actual data from OpenWeatherMap and co-ordinating day/night icon with UTC instead of local time. None of these bugs are there in Cinnamon’s weather plugin.

Appearance: Themes and Wallpapers

We will install the following themes and wallpapers:

To apply this theme select “Numix” in

• Settings Manager > Window Manager > Style
• Settings Manager > Appearance > Style
• Settings Manager > Appearance > Icons
• Settings Manager > LightDM GTK+ Greeter settings > Appearance

Note that the default appearance editor can only edit GTK2 themes/icons. Therefore, to edit GTK3 themes/icons either edit the ~/.config/gtk-3.0/settings.ini or use the LXDE appearance editor lxappearance. You might also have to make changes in Settings Manager > Windows Manager Tweaks > Accessibility.

Finally, to change background use “Settings Manager > Desktop > Background” and select the folder /usr/share/backgrounds/f32/extras/.

Introspection

The plan is to upgrade to next version of Fedora three months after its release so that maximum bugs can be avoided (eg: initial release of Fedora 32 had a bug which has been corrected later using system updates). Hence I will get a system wide update twice a year. Fedora is going to change to btrfs as default filesystem in the next release of Fedora 33 (something default in SUSE but abandoned by Red Hat). One annoying thing is that I have to install lots of updates everyday since Fedora pushes bug fixes as soon as they are found, irrespective of how important they are. Moreover, the supported releases are updated to the latest stable version of the Linux kernel, increasing the chances of unstability.

Though Xfce is highly customizable and has a great documentation, it lacks GUI for basic things like system info viewer, individual CPU core usage, user login picture manager (though Fedora has Whisker menu plugin, it lacks the required Mugshot package) and tablet pen-pressure settings (which can also be done in GNOME 3, Cinnamon and KDE). However, these can easily be done using manually via Terminal:

• For system info we have CLI tools like inxi (doc), neofetch and lshw.

• To view system usage information we can use tool like top in terminal, and get individual core usage by top 1 (doc).

• We can manually edit the profile picture configuration

$ sudo mkdir /var/lib/AccountsService/icons/<your_user_name>
$ sudo cp ~/home/<my_picture_of_size_96x96>.png /var/lib/AccountsService/icons/<your_user_name>

$ sudo mousepad /var/lib/AccountsService/users/<your_user_name>

Add the line: Icon=/var/lib/AccountsService/icons/<your_user_name>/<my_picture_of_size_96x96>.png.

• For Wacom tablet pen-pressure on X Window System we can use xsetwacom (doc)

xsetwacom set "Wacom One by Wacom S Pen stylus" PressureCurve 0 50 50 100

However, any setting changed by xsetwacom will be reset to default (or a statically configured setting) whenever the device is unplugged, disabled or the X Server is restarted. One can make Linux run the command automatically when hot-plugging the device by entering the above command to Settings Manager > Removable Drives and Media > Input Devices > Automatically run this command when a tablet is connected. Note that, these CLI methods won’t work when using Wayland instead of Xorg. I could’t make CLI work in Cinnamon, only GUI settings worked and provided the following options:

  0  75   25  100  # very soft
  0  50   50  100  # soft
  0  25   75  100  # little soft
  0   0  100  100  # default
 25   0  100   75  # little firm
 50 100  100   50  # firm
 75   0  100   25  # very firm

Though main advantage of Fedora is that it is much more customizable. However, the following three errors were still logged:

1. AMD-Vi: Most likely this is a false alarm from the Linux kernel. However, turning off IOMMU in BIOS is not an option (which somehow is suggested in forums) since it is needed for HSA used by AMD for making GPU and CPU work together in its APUs. If I end up with black-screen during boot then I will consider updating AGESA drivers in the BIOS.

2. Watchdog: Another AMD Ryzen related bug in Linux Kernel. We can just silence it. If I end up with black-screen during boot then I will consider updating AGESA drivers in the BIOS.

3. GNOME key-ring: Issue originating from GNOME which have crept into LightDM.

Epilogue

After one year of using Fedora 32 and 33 with Xfce 4.14, in June 2021, I moved to OpenSUSE Leap 15.3 with KDE 5.18 instead of Fedora 34 with Xfce/GNOME. It was mainly because of the issues with Xfce 4.16 (unable to control power settings for idle time) and Linux kernel 5.12 (messing with swap partition, hanging since not freeing the RAM). I was also annoyed by the large number of updates I received everyday on Fedora (but stuck with TexLive 2017, which was always up-to-date in Fedora). However, I do miss the package information system and documentation of Fedora. Other than that, OpenSUSE and KDE have worked great out of the box (DON’T ENCRYPT THE HARD DRIVE, disable secureboot and disable NVRAM). I had to install OpenSUSE in virtual machine first, in order to make the installation USB using SUSE Studio Image Writer (moreover, change in college internet connection policy made it impossible to use netinstall). Another option would be to use Ventoy.

After installing, I used windows to reformat the USB stick to exFAT format. Then to access USB stick in openSUSE, install fuse-exfat package.

If Konsole starts in super user mode, then go to Settings > Manage Profiles > Edit profile > General it is important that in ‘Command’ it does not say su- but /bin/bash.

Disable automatic screen locking: System Settings > Workspace behavior > screen locking

Adjust screen off time: System Settings > Power management > energy saving

Disable the auto-updater PackageKit since it can interfere with the manual software installation: Configure System Tray > Software Updates > Disabled.

Replace Firefox’s Extended Support Release (ESR) with the latest version from the Mozilla repository using YaST or terminal:

> sudo zypper ar -f http://download.opensuse.org/repositories/mozilla/openSUSE_Leap_15.3/ mozilla
> sudo zypper ref
> zypper up --from mozilla

Unfortunately, the fonts don’t render properly sometimes due to KDE integration limitations. Also install Plasma integration add-on. Then, to watch Netflix, also install the KDE multimedia codecs from Packman using 1-click. Somehow, vlc-lang package didn’t swtich the repository so had to do it manually.

> sudo zypper up --allow-vendor-change
Loading repository data...
Reading installed packages...

The following package is going to be upgraded:
  vlc-lang

The following package is going to change vendor:
  vlc-lang  openSUSE -> http://packman.links2linux.de

Also, had to build SageMath from the source, took 4 hours (for reference: 6000+ packages of texlive-scheme-full take 1 hour to download and install)

> sudo zypper install gcc gcc-c++ make m4 binutils perl tar python3 dvipng ffmpeg texlive-scheme-full

> ./configure
# we will skip installing the suggested packages, openSUSE Leap has a limted collection of packages.
# will rather build everything from source, hence the long build time.

> make
-----
[2021-07-14 22:12:09] SageMath version 9.3, Release Date: 2021-05-09
Yes, Sage starts.

real    225m36.799s
user    229m31.133s
sys     13m38.360s
Sage build/upgrade complete!
-----

> ./sage --testall

> sudo ln -s /path/to/sage-x.y/sage /usr/local/bin/sage

There is color-scheme bug. Ceate a file called 50-color.ipy in .sage/ipython-X.Y.Z/profile_default/startup with the following content:

%colors Linux

Also, KDE icon can be created using the folliwing command for launching in Applications tab:

konsole -T "sage" -e /path/to/sage-x.y/sage

Performance

Following is the comparison of my PC with the best laptop available in the market in May 2020 for USD 650 (before tax):

This will certainly an upgrade from the desktop computer I used a decade ago with Intel Pentium 4 CPU and 512MB RAM (no idea what graphics card was there in that machine). Moreover, though on paper Intel Core i7 4702MQ (my old Thinkpad) is not much worse than AMD Ryzen 3 3200G in terms of CPU performance, the AMD iGPU much better. For instance, with the best graphic settings of SuperTuxKart (for 60FPS/1080p required to have min AMD Radeon RX 460 with 1 GB VRAM), AMD gives about 30 fps on an average whereas Intel only gives 10 fps.

Cloud Backup Setup for Linux PC (tested in Linux Mint and Ubuntu)

I am using rclone to sync files between my PC and Google Drive. Following are the steps one can follow to set it up:

1. Create your own Google Drive OAuth2 client ID for rclone:

   a) Log into the Google API Console with your Google account. It doesn’t matter what Google account you use. (It need not be the same account as the Google Drive you want to access.

   b) Select a project or create a new project.

   c) Under “ENABLE APIS AND SERVICES” search for “Drive”, and enable the “Google Drive API”.

   d) Click “Oauth Consent Screen” in the left panel and select user type “External”. Then add Application name (anything you want) and save.

   e) Click “Credentials” in the left panel. Then click on “+ CREATE CREDENTIALS” button at the top of the screen, then select “OAuth client ID”. Select Application type as “Desktop app”, enter whaever client anme you want and click create.

   f) It will show you a client ID and client secret. Use these values in rclone config.

2. Install the latest version of rclone via Terminal: curl https://rclone.org/install.sh | sudo bash
3. Now configure rclone for Google Drive via Terminal: rclone config and follow the steps. Remember to use the Client ID and client secret we created above. Since we created this API for personal use, we won’t be submitting it for verfication. Hence don’t be alarmed by the very scary confirmation screen shown when we connect via your browser for rclone to be able to get its token-id. Also, if you want to fetch Google Docs as links (instead of converting them to .odt etc) from Google Drive, in “advance-config” set “export-formats” to “link.html”.
4. Sync files using “copy” and NOT “sync”: rclone copy source:path dest:path [flags]. For example, to sync all files from “New Folder” Google Drive (named: Drive) to PC (folder: home) and view the progress, type: rclone copy Drive:"New Folder" /home -P. Google Drive tend to have duplicate files since it allows same names files in same folder, in that case use dedupe to delete all duplicate files.

5. Set-up auto update for rclone program (can see the latest version here):

   a) Open the bash file using a text editor like gedit/xed/vim: ~/.bashrc

   b) Add the following code to the file and save it: rclone() { if [[ $@ == "-U" ]]; then command curl https://rclone.org/install.sh | sudo bash; else command rclone "$@"; fi; }

   c) To update to the latest version run this in the Terminal: rclone -U and enter the PC password.

In case you wish to remove rclone, run the following commands in terminal which will delete the files containing information about rclone:

sudo rm /usr/bin/rclone
sudo rm /usr/local/share/man/man1/rclone.1

Build Details

All the system part recommendations were from PC-Part-Picker forums and Build-a-PC subreddit hence were biased towards gaming performance.

I also bought IKEA Micke (Black-Brown) computer desk for kids (had to remove the bigger drawer to accomodate my legs) and IKEA Flintan+Nominell office chair (USD 80 + USD 100).

Introspection

The original plan was to use Ubuntu 20.04 GNOME as the OS since I wanted a stable system (hence not the rolling release distros like Arch/Manjaro, Solus and openSUSE TW) with Linux kernel greater than 4.20 for AMD Ryzen Picasso APUs (hence not Debian and CentOS) and large software repository (hence not enterprise focused distros like OpenSUSE Leap and Fedora). But Ubuntu turned out to be RAM hungry, since it required extra softwares like GNOME extensions and Variety for customization, and depended on the unstable Snap store. While trying to fix these issues I was able to brick my system thrice and hence decided to shift to Linux Mint 20 Cinnamon which provides out-of-the-box support for Flatpak(needed for Xournal++) and blocks Snap while providing better customizability and the same 5 year support (unlike official Ubuntu flavours which give only 3 years support). Though the appearance and RAM usage became much better with Linux Mint, it still inherited many boot errors like:

1. IOMMU: Most likely this is a false alarm from the Linux kernel. However, turning off IOMMU in BIOS is not an option (which somehow is suggested in forums) since it is needed for HSA used by AMD for making GPU and CPU work together in its APUs. If I end up with black-screen during boot then I will consider updating AGESA drivers in the BIOS.

2. Time Stamp Counter: Another Ryzen APU compatibility issue with Linux kernel.

3. Kwallet: KWallet is supposed to be with KDE and not Cinnamon/GNOME.

4. GNOME key-ring: Issue with GNOME.

5. Pulseaudio+BlueZ: due to unavailability of Bluetooth in my computer and presence of Pulseaudio and BlueZ 5.53 in Ubuntu 20.04.

6. initramfs: Boot speed improvements through changing the default kernel compression algorithm to lz4 (in Ubuntu 19.10) on most architectures, and changing the default initramfs compression algorithm to lz4 on all architectures. Somehow doesn’t work well for my PC. I switched to gzip compression to avoid this bug.

7. UVC driver/Alsa: I think that the audio driver of Webcam leading to error during boot.

Out of these 1 and 4 are logged consistently during each boot, 2 and 7 are logged some times, and the remaining could be fixed if they ever occur again. Since I am running LTS distro, I won’t have to worry much about losing compaibility of my hardware or learning about latest softwares. Though there are only non-fatal boot errors, since everything is working as intended, it might be best to just shrug sholders.

Also, if you are using NVMe SSD then consider installing the NVMe tool as discussed here (might lead to logging of boot errors). Once you install any linux OS I would recommend running following commands and checking is everything is running as expected:

inxi -Fxz #system information google to learn more about inxi

lscpu #can also see lspci, lshw

sudo nvme smart-log /dev/nvme0n1" #NVMe info

journalctl -b -p 3 #error log messages

dmesg -l emerg,alert,crit,err

Also, in case things get out of hand, to access GRUB Menu press right Shift if system boots using BIOS or press Esc if system boots using UEFI. Remeber that if you press Esc multiple times then you will enter GRUB command prompt instead.

The current AMD budget processors provide a much superior iGPU performance than their Intel counterparts. For example:

• We can compare the entry level quad-core desktop processors from last year: Intel Core i3-9100 (MSRP USD 120) has better CPU but worse iGPU than AMD Ryzen 3 3200g (MSRP USD 100) (benchmark).
• We can compare the pro level mobile processors from this year: Intel Core i7-10710U with 6-physical cores and hyperthreading, is not better than AMD Ryzen 7 4700U with 8-physical cores but no hyperthreading (comparison).

I decided not to spend extra USD 20 for i3-9100 since the ability of 3200g to allow light gaming would be more than enough for my daily usage. Moreover, since neither of them have hyperthereading/multithreading, I won’t expect much improvement in multitasking. However, Intel being more popular has better support for various hardware (for example, I couldn’t run my monitor via HDMI due to compatibility issue between the motherboard and monitor). Luckily, this time the Realtek ethernet drivers didn’t cause headache with Ubuntu.

We can’t deny the fact that, Intel Core i3 9100 would have been a better choice in terms of processing power. Even after the launch of 10th gen CPUs that build would have costed about USD 20 more since though the RAM will be really cheap as only 2400 MHz is supported by i3 (- USD 10), the CPU and motherboard (H310) will remain a little more expensive than the AMD counterparts (+ USD 30). however, by buying HP VH240a instead of Asus VA229HR, I could have saved USD 30, hence keeping the total cost of build USD 650 even with i3 9100. That said, I am happy with my AMD PC since it allows me to do some light gaming as a bonus (which was not the purpose of this PC). AMD build is clearly better than Intel if one plans to play even a single racing/shooting game with an option for overclocking by buying a better motherboard.