The following outline is provided as an overview of, and topical guide to, deep learning: Deep learning is a subfield of machine learning and artificial intelligence based on artificial neural networks with multiple processing layers. It emphasizes representation learning and is widely used in areas such as computer vision, natural language processing, speech recognition, recommender systems, robotics, and generative artificial intelligence.
Ways to categorize deep learning A field of study A branch of artificial intelligence A subfield of machine learning A subfield of computer science A form of representation learning A class of methods based on artificial neural networks An approach used in computational statistics
History
Precursors Cybernetics Perceptron Connectionism Neocognitron Backpropagation
Milestones LeNet Long short-term memory Deep belief network AlexNet Sequence to sequence learning Generative adversarial network Residual neural network Transformer BERT Generative pre-trained transformer Diffusion model
Related histories History of artificial intelligence History of machine learning Timeline of machine learning
Core concepts
Learning settings Supervised learning Unsupervised learning Self-supervised learning Semi-supervised learning Reinforcement learning Transfer learning Multitask learning Multimodal learning Online machine learning Continual learning
Common tasks Image classification Object detection Image segmentation Automatic speech recognition Neural machine translation Question answering Automatic summarization Text-to-image model Protein structure prediction
Architectures
Feedforward and convolutional architectures Feedforward neural network Multilayer perceptron Convolutional neural network Radial basis function network Residual neural network U-Net
Recurrent and sequence architectures Recurrent neural network Long short-term memory Gated recurrent unit Sequence to sequence learning Recursive neural network
Representation-learning architectures Autoencoder Denoising autoencoder Sparse autoencoder Variational autoencoder Restricted Boltzmann machine Deep belief network
Attention and transformer architectures Attention (machine learning) Transformer BERT Generative pre-trained transformer Vision transformer
Generative and probabilistic architectures Autoregressive model Diffusion model Energy-based model Generative adversarial network Mixture of experts
Graph and memory architectures Graph neural network Graph convolutional network Siamese network Neural Turing machine Memory network Echo state network Capsule neural network
Neural network components and techniques
Artificial neuron Activation function Rectified linear unit Sigmoid function Softmax function Embedding Convolution Pooling layer Attention Batch normalization Layer normalization Residual connections
Training and optimization Backpropagation Gradient descent Stochastic gradient descent Adam optimization Learning rate Loss function Cross-entropy Mean squared error Regularization Dropout Early stopping Batch normalization Data augmentation Transfer learning Knowledge distillation Ensemble learning Curriculum learning
Datasets and benchmarks CIFAR-10 ImageNet MNIST database Common Objects in Context (COCO) General Language Understanding Evaluation (GLUE) benchmark LibriSpeech SQuAD
Applications
Computer vision Computer vision Facial recognition system Image classification Image segmentation Medical imaging Object detection Optical character recognition
Natural language processing Automatic summarization Chatbot Information retrieval Large language model Natural language processing Neural machine translation Question answering Sentiment analysis
Speech and audio
Automatic speech recognition Music information retrieval Speaker recognition Speech synthesis
Science and medicine Bioinformatics Computational biology Drug discovery Medical diagnosis Protein structure prediction
Robotics and control Autonomous car Computer game bot Control theory Robotics
Recommendation, search, and forecasting Anomaly detection Forecasting Fraud detection Recommender system Search engine
Generative artificial intelligence Deepfake Generative artificial intelligence Large language model Speech synthesis Text-to-image model
Computer graphics and video games Deep Learning Anti-Aliasing (DLAA) Deep Learning Super Sampling (DLSS)
Hardware AMD Instinct AMD XDNA Application-specific integrated circuit Deep learning processor, Neural processing unit (NPU), or Neural Engine Field-programmable gate array General-purpose computing on graphics processing units (GPGPU) Graphics processing unit NVIDIA Deep Learning Accelerator (NVDLA) Tensor processing unit Vision processing unit Wafer-scale integration
Supporting software platforms CUDA Metal ROCm
Software
Open-source frameworks and libraries
Neural network software
EDLUT Emergent Encog JOONE Neuroph NeuroSolutions OpenNN Peltarion Synapse SNNS
Platforms, tools, and deployment Amazon SageMaker Google Colab Hugging Face Kaggle Kubeflow MLflow ONNX OpenVINO TensorFlow Hub
Algorithms for deep learning and neural networks
Backpropagation Conjugate gradient method Generalized Hebbian algorithm Gradient descent Levenberg–Marquardt algorithm Perceptron Quasi-Newton method Wake-sleep algorithm
Methods and related topics
Representation and metric learning Contrastive learning Embedding Feature learning Manifold learning Metric learning
Generative modeling Autoregressive model Diffusion model Generative adversarial network Generative model Variational inference
Efficient and scalable deep learning Knowledge distillation Low-rank approximation Mixture of experts Quantization Sparsity
Reliability, safety, and interpretability Adversarial machine learning AI alignment Algorithmic bias Catastrophic forgetting Differential privacy Explainable artificial intelligence Federated learning Hallucination (artificial intelligence)
Conferences and workshops Annual Meeting of the Association for Computational Linguistics Conference on Computer Vision and Pattern Recognition Conference on Neural Information Processing Systems International Conference on Computer Vision International Conference on Learning Representations International Conference on Machine Learning
Organizations
Research laboratories and institutions Allen Institute for AI Alberta Machine Intelligence Institute European Laboratory for Learning and Intelligent Systems Google DeepMind Meta AI Mila Microsoft Research Vector Institute
… excerpt ends here. Continue reading the full article.
