[Table of Contents]

Preface
Introduction

A Machine Learning and Physics

A1. Linear Models
　A1.1 Least Squares Method and Linear Regression
　　A1.1.1 Least Squares Method
　　A1.1.2 Convex Functions
　　A1.1.3 Conditions for Convexity of Multivariate Functions
　　A1.1.4 Linear Models
　　A1.1.5 Continuation of Least Squares Method
　A1.2 Entropy
　　A1.2.1 Probability
　　A1.2.2 Shannon Entropy
　　A1.2.3 Relative Entropy and KL Divergence
　　A1.2.4 Jensen's Inequality
　　A1.2.5 Gaussian Distribution
　A1.3 Maximum Likelihood Estimation
　　A1.3.1 Likelihood Function
　　A1.3.2 Maximum Likelihood from KL Divergence
　A1.4 Generalized Linear Models
　　A1.4.1 Binary Classification and Logistic Regression
　　A1.4.2 Origin of Cross-Entropy
　A1.5 Classification of Machine Learning
　A1.6 Generalization, Overfitting and Underfitting
　A1.7 Random Numbers
　　A1.7.1 What are Random Numbers
　　A1.7.2 Uniform Random Numbers
　　A1.7.3 Gaussian Random Numbers

A2. Neural Networks (NN)
　A2.1 Neural Networks
　A2.2 Data Representation
　　A2.2.1 Vectorization of Images
　　A2.2.2 One-Hot Representation
　A2.3 Fully Connected Neural Networks with General Number of Layers
　A2.4 Gradient Descent Method
　A2.5 Activation Functions and Their Derivatives
　A2.6 Backpropagation
　A2.7 Gradient Vanishing Problem

A3. Symmetry and Machine Learning: Convolution and Equivariant NN
　A3.1 Equivariance and Convolutional Neural Networks
　A3.2 Image Filters
　A3.3 Convolutional Layer
　　A3.3.1 Two-Dimensional Convolution
　　A3.3.2 Pooling
　A3.4 Group Theory and Symmetry
　A3.5 Symmetry and Equivariance
　　A3.5.1 Ways to Incorporate Symmetry
　　A3.5.2 Group Equivariant Neural Networks
　　A3.5.3 Inductive Bias
　　A3.5.4 Gauge Symmetry and Neural Networks
　　
A4. Classical Mechanics and Machine Learning: Neural Networks and Differential Equations
　A4.1 Fundamental Equations of Physics and Machine Learning
　　A4.1.1 The Role of Differential Equations
　　A4.1.2 Embedding Physics Problems into Machine Learning
　A4.2 Physics-Informed Neural Networks (PINN)
　A4.3 Viewing Neural Networks as Differential Equations
　　A4.3.1 Methods for Handling Differential Equations in Machine Learning
　　A4.3.2 Locality of NN
　　A4.3.3 ResNet and Differential Equations
　　A4.3.4 Locality within Layers and Convolutional NN
　A4.4 Representation of Specific Equations of Motion by NN
　　A4.4.1 Example of a Particle in a Potential
　　A4.4.2 Hamiltonian Systems

A5. Quantum Mechanics and Machine Learning: Neural Network Wave Functions
　A5.1 Quantum Mechanics and Eigenvalue Problems
　A5.2 Quantum Many-Body Problems on Lattices
　A5.3 Variational Method and Trial Functions
　A5.4 Neural Network Wave Functions in Small Quantum Systems
　　A5.4.1 Analytical Solution of the Two-Site Transverse-Field Ising Model
　　A5.4.2 Approximate Solution Using Neural NetworkWave Functions
　A5.5 Neural Network Wave Functions in Larger Quantum Systems
　　A5.5.1 Exact Numerical Solution Using the Exact Diagonalization Method
　　A5.5.2 Approximate Numerical Solution Using Neural Network Wave Function
　A5.6 Future Prospects
　
B Machine Learning Models and Physics

B1. Transformer
　B1.1 Words and Embedding Vectors
　　B1.1.1 Use Theory of Meaning and Embedding
　　B1.1.2 Search from Key-Value Store and Attention Mechanism
　　B1.1.3 Transformer Architecture
　B1.2 Transformers in NLP and Computer Vision
　　B1.2.1 GPT
　　B1.2.2 Vision Transformer
　　
B2. Diffusion Models and Path Integrals
　B2.1 Principles of Diffusion Models
　　B2.1.1 The Idea of Diffusion Models
　　B2.1.2 Diffusion Models and Langevin Equation
　　B2.1.3 Sampling Process of Diffusion Models
　　B2.1.4 Training of Diffusion Models
　　B2.1.5 Probability Flow ODE
　B2.2 Path Integral Quantization
　B2.3 Path Integral Formulation of Diffusion Models
　　B2.3.1 Derivation of the Reverse Process
　　B2.3.2 Derivation of Loss Function for Diffusion Model Training
　　B2.3.3 Probability Flow and Classical Limit

B3. Mechanism Behind Machine Learning: Statistical Mechanical Approach
　B3.1 Infinite-Width DNN: Signal Propagation
　　B3.1.1 Spin Model
　　B3.1.2 Macroscopic Laws of Signal Propagation
　　B3.1.3 Mean Field Theory and Order-to-Chaos Phase Transition
　　B3.1.4 Macroscopic Law of Backpropagation
　　B3.1.5 Vanishing and Exploding Gradient Problem as Phase Transition
　　B3.1.6 Connection with Kernel Methods
　B3.2 Infinite-Width DNN Model: Learning Regimes
　　B3.2.1 NTK Regime
　　B3.2.2 μP
　B3.3 Linear Regression Model
　　B3.3.1 Generalization Error in Over-Parameterized Models
　　B3.3.2 Typical Evaluation of Generalization Error
　　
B4. Large Language Models and Science
　B4.1 Large Language Models
　　B4.1.1 Next Word Prediction
　　B4.1.2 Training of Large Language Models
　B4.2 Applications of Large Language Models
　　B4.2.1 Arithmetic Capabilities of Large Language Models
　　B4.2.2 Proof Capabilities of Large Language Models
　　B4.2.3 Cubism in Mathematics

Afterword
Index