logo资料库

Deep Reinforcement Learning Hands-On Apply modern RL methods,.pdf

发布时间：2022-05-31 发布人：admin 分类：说明书 资料大小：12.81M 资料格式：pdf 举报 版权申诉
第1页 / 共752页
第2页 / 共752页
第3页 / 共752页
第4页 / 共752页
第5页 / 共752页
第6页 / 共752页
第7页 / 共752页
第8页 / 共752页
资料共752页，剩余部分请下载后查看
    Deep Reinforcement Learning Hands-On
    Why subscribe?
    PacktPub.com
    Contributors
    About the author
    About the reviewers
    Packt is Searching for Authors Like You
    Preface
    Who this book is for
    What this book covers
    To get the most out of this book
    Download the example code files
    Download the color images
    Conventions used
    Get in touch
    Reviews
    1. What is Reinforcement Learning?
    Learning – supervised, unsupervised, and reinforcement
    RL formalisms and relations
    Reward
    The agent
    The environment
    Actions
    Observations
    Markov decision processes
    Markov process
    Markov reward process
    Markov decision process
    Summary
    2. OpenAI Gym
    The anatomy of the agent
    Hardware and software requirements
    OpenAI Gym API
    Action space
    Observation space
    The environment
    Creation of the environment
    The CartPole session
    The random CartPole agent
    The extra Gym functionality – wrappers and monitors
    Wrappers
    Monitor
    Summary
    3. Deep Learning with PyTorch
    Tensors
    Creation of tensors
    Scalar tensors
    Tensor operations
    GPU tensors
    Gradients
    Tensors and gradients
    NN building blocks
    Custom layers
    Final glue – loss functions and optimizers
    Loss functions
    Optimizers
    Monitoring with TensorBoard
    TensorBoard 101
    Plotting stuff
    Example – GAN on Atari images
    Summary
    4. The Cross-Entropy Method
    Taxonomy of RL methods
    Practical cross-entropy
    Cross-entropy on CartPole
    Cross-entropy on FrozenLake
    Theoretical background of the cross-entropy method
    Summary
    5. Tabular Learning and the Bellman Equation
    Value, state, and optimality
    The Bellman equation of optimality
    Value of action
    The value iteration method
    Value iteration in practice
    Q-learning for FrozenLake
    Summary
    6. Deep Q-Networks
    Real-life value iteration
    Tabular Q-learning
    Deep Q-learning
    Interaction with the environment
    SGD optimization
    Correlation between steps
    The Markov property
    The final form of DQN training
    DQN on Pong
    Wrappers
    DQN model
    Training
    Running and performance
    Your model in action
    Summary
    7. DQN Extensions
    The PyTorch Agent Net library
    Agent
    Agent's experience
    Experience buffer
    Gym env wrappers
    Basic DQN
    N-step DQN
    Implementation
    Double DQN
    Implementation
    Results
    Noisy networks
    Implementation
    Results
    Prioritized replay buffer
    Implementation
    Results
    Dueling DQN
    Implementation
    Results
    Categorical DQN
    Implementation
    Results
    Combining everything
    Implementation
    Results
    Summary
    References
    8. Stocks Trading Using RL
    Trading
    Data
    Problem statements and key decisions
    The trading environment
    Models
    Training code
    Results
    The feed-forward model
    The convolution model
    Things to try
    Summary
    9. Policy Gradients – An Alternative
    Values and policy
    Why policy?
    Policy representation
    Policy gradients
    The REINFORCE method
    The CartPole example
    Results
    Policy-based versus value-based methods
    REINFORCE issues
    Full episodes are required
    High gradients variance
    Exploration
    Correlation between samples
    PG on CartPole
    Results
    PG on Pong
    Results
    Summary
    10. The Actor-Critic Method
    Variance reduction
    CartPole variance
    Actor-critic
    A2C on Pong
    A2C on Pong results
    Tuning hyperparameters
    Learning rate
    Entropy beta
    Count of environments
    Batch size
    Summary
    11. Asynchronous Advantage Actor-Critic
    Correlation and sample efficiency
    Adding an extra A to A2C
    Multiprocessing in Python
    A3C – data parallelism
    Results
    A3C – gradients parallelism
    Results
    Summary
    12. Chatbots Training with RL
    Chatbots overview
    Deep NLP basics
    Recurrent Neural Networks
    Embeddings
    Encoder-Decoder
    Training of seq2seq
    Log-likelihood training
    Bilingual evaluation understudy (BLEU) score
    RL in seq2seq
    Self-critical sequence training
    The chatbot example
    The example structure
    Modules: cornell.py and data.py
    BLEU score and utils.py
    Model
    Training: cross-entropy
    Running the training
    Checking the data
    Testing the trained model
    Training: SCST
    Running the SCST training
    Results
    Telegram bot
    Summary
    13. Web Navigation
    Web navigation
    Browser automation and RL
    Mini World of Bits benchmark
    OpenAI Universe
    Installation
    Actions and observations
    Environment creation
    MiniWoB stability
    Simple clicking approach
    Grid actions
    Example overview
    Model
    Training code
    Starting containers
    Training process
    Checking the learned policy
    Issues with simple clicking
    Human demonstrations
    Recording the demonstrations
    Recording format
    Training using demonstrations
    Results
    TicTacToe problem
    Adding text description
    Results
    Things to try
    Summary
    14. Continuous Action Space
    Why a continuous space?
    Action space
    Environments
    The Actor-Critic (A2C) method
    Implementation
    Results
    Using models and recording videos
    Deterministic policy gradients
    Exploration
    Implementation
    Results
    Recording videos
    Distributional policy gradients
    Architecture
    Implementation
    Results
    Things to try
    Summary
    15. Trust Regions – TRPO, PPO, and ACKTR
    Introduction
    Roboschool
    A2C baseline
    Results
    Videos recording
    Proximal Policy Optimization
    Implementation
    Results
    Trust Region Policy Optimization
    Implementation
    Results
    A2C using ACKTR
    Implementation
    Results
    Summary
    16. Black-Box Optimization in RL
    Black-box methods
    Evolution strategies
    ES on CartPole
    Results
    ES on HalfCheetah
    Results
    Genetic algorithms
    GA on CartPole
    Results
    GA tweaks
    Deep GA
    Novelty search
    GA on Cheetah
    Results
    Summary
    References
    17. Beyond Model-Free – Imagination
    Model-based versus model-free
    Model imperfections
    Imagination-augmented agent
    The environment model
    The rollout policy
    The rollout encoder
    Paper results
    I2A on Atari Breakout
    The baseline A2C agent
    EM training
    The imagination agent
    The I2A model
    The Rollout encoder
    Training of I2A
    Experiment results
    The baseline agent
    Training EM weights
    Training with the I2A model
    Summary
    References
    18. AlphaGo Zero
    Board games
    The AlphaGo Zero method
    Overview
    Monte-Carlo Tree Search
    Self-play
    Training and evaluation
    Connect4 bot
    Game model
    Implementing MCTS
    Model
    Training
    Testing and comparison
    Connect4 results
    Summary
    References
    Book summary
    Other Books You May Enjoy
    Leave a review - let other readers know what you think
    Index
    Deep Reinforcement Learning Hands-On
    Table of Contents Deep Reinforcement Learning Hands-On Why subscribe? PacktPub.com Contributors About the author About the reviewers Packt is Searching for Authors Like You Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Conventions used Get in touch Reviews 1. What is Reinforcement Learning? Learning – supervised, unsupervised, and reinforcement RL formalisms and relations Reward The agent The environment Actions Observations Markov decision processes Markov process Markov reward process Markov decision process Summary 2. OpenAI Gym The anatomy of the agent Hardware and software requirements OpenAI Gym API
    Action space Observation space The environment Creation of the environment The CartPole session The random CartPole agent The extra Gym functionality – wrappers and monitors 3. Deep Learning with PyTorch Wrappers Monitor Summary Tensors Creation of tensors Scalar tensors Tensor operations GPU tensors Gradients Tensors and gradients NN building blocks Custom layers Final glue – loss functions and optimizers Monitoring with TensorBoard Loss functions Optimizers TensorBoard 101 Plotting stuff Example – GAN on Atari images Summary 4. The Cross-Entropy Method Taxonomy of RL methods Practical cross-entropy Cross-entropy on CartPole Cross-entropy on FrozenLake Theoretical background of the cross-entropy method Summary 5. Tabular Learning and the Bellman Equation Value, state, and optimality
    Interaction with the environment SGD optimization Correlation between steps The Markov property The final form of DQN training DQN on Pong Wrappers DQN model Training Running and performance Your model in action Summary 7. DQN Extensions The PyTorch Agent Net library The Bellman equation of optimality Value of action The value iteration method Value iteration in practice Q-learning for FrozenLake Summary 6. Deep Q-Networks Real-life value iteration Tabular Q-learning Deep Q-learning Agent Agent's experience Experience buffer Gym env wrappers Basic DQN N-step DQN Implementation Double DQN Implementation Results Noisy networks Implementation Results
    Prioritized replay buffer Implementation Results Dueling DQN Implementation Results Categorical DQN Implementation Results Combining everything Implementation Results Summary References 8. Stocks Trading Using RL Trading Data Problem statements and key decisions The trading environment Models Training code Results The feed-forward model The convolution model 9. Policy Gradients – An Alternative Things to try Summary Values and policy Why policy? Policy representation Policy gradients The REINFORCE method The CartPole example Results Policy-based versus value-based methods REINFORCE issues Full episodes are required
    High gradients variance Exploration Correlation between samples PG on CartPole Results PG on Pong Results Summary 10. The Actor-Critic Method Variance reduction CartPole variance Actor-critic A2C on Pong A2C on Pong results Tuning hyperparameters Learning rate Entropy beta Count of environments Batch size Summary 11. Asynchronous Advantage Actor-Critic Correlation and sample efficiency Adding an extra A to A2C Multiprocessing in Python A3C – data parallelism A3C – gradients parallelism Results Results Summary 12. Chatbots Training with RL Chatbots overview Deep NLP basics Recurrent Neural Networks Embeddings Encoder-Decoder Training of seq2seq Log-likelihood training
    Bilingual evaluation understudy (BLEU) score RL in seq2seq Self-critical sequence training The chatbot example The example structure Modules: cornell.py and data.py BLEU score and utils.py Model Training: cross-entropy Running the training Checking the data Testing the trained model Training: SCST Running the SCST training Results Telegram bot Summary 13. Web Navigation Web navigation Browser automation and RL Mini World of Bits benchmark OpenAI Universe Installation Actions and observations Environment creation MiniWoB stability Simple clicking approach Grid actions Example overview Model Training code Starting containers Training process Checking the learned policy Issues with simple clicking Human demonstrations Recording the demonstrations
    分享到：
    赞 收藏

    相关推荐

    资料库

    人工智能

    共收录2047份资料，累计62个分类，关注成员有19位，主要包括：GNSS,Pico-SIM868-GSM,GPRS,NB-IoT,GPRSHAT,SIM7070GCat-M,GNSSHAT,GSM,NB-IoTHAT,SIM7080GCat-M,R800CGSM,SIM800CGSM,UPSPowerModule(B),UPSPowerModule,Pico-SIM7020C-NB-IoT,SIM7028NB-IoTHAT,SIM7020CNB-IoTHAT,SIM7000XNB-IoTHAT,A7670CCat-1HAT,A7600C1Cat-1,SIM7600X4G&LTECat-1HAT,SIM7600X4GDONGLE用户手册,RM520N-GL5GHAT,RM500Q-GL5GHAT,RM500U-CN5GHAT,SIM820X-M25GforJetsonNano,AudioCardforJetsonNano,SIM7600G-H4GforJetsonNano,OpenNCC-Lite-4G8M,IMX219-77Camera,13.3inche-PaperHAT,10.3inche-PaperHAT(D),10.3inche-PaperHAT,9.7inche-PaperHAT,7.8inche-PaperHAT,6inchHDTouche-PaperHAT,6inchHDe-PaperHAT,6inche-PaperHAT,5.83inche-PaperHAT(C),LCD-Clock-A,0.95inchRGBOLED(A),0.96inchRGBOLEDModule,0.96inchOLEDModule,0.96inchOLED(A),1.27inchRGBOLEDModule,1.3inchOLEDModule(C),1.3inchOLED(A),1.32inchOLEDModule,1.5inchRGBOLEDModule,1.5inchOLEDModule,JETSONORINNX,自然语言处理,搜索引擎,VR,计算广告,深度学习,机器学习

    热门标签

    GNSS Pico-SIM868-GSM GPRS NB-IoT GPRSHAT SIM7070GCat-M GNSSHAT GSM NB-IoTHAT SIM7080GCat-M R800CGSM SIM800CGSM UPSPowerModule(B) UPSPowerModule Pico-SIM7020C-NB-IoT SIM7028NB-IoTHAT SIM7020CNB-IoTHAT SIM7000XNB-IoTHAT A7670CCat-1HAT A7600C1Cat-1 SIM7600X4G&LTECat-1HAT SIM7600X4GDONGLE用户手册 RM520N-GL5GHAT RM500Q-GL5GHAT RM500U-CN5GHAT SIM820X-M25GforJetsonNano AudioCardforJetsonNano SIM7600G-H4GforJetsonNano OpenNCC-Lite-4G8M IMX219-77Camera 13.3inche-PaperHAT 10.3inche-PaperHAT(D) 10.3inche-PaperHAT 9.7inche-PaperHAT 7.8inche-PaperHAT 6inchHDTouche-PaperHAT 6inchHDe-PaperHAT 6inche-PaperHAT 5.83inche-PaperHAT(C) LCD-Clock-A 0.95inchRGBOLED(A) 0.96inchRGBOLEDModule 0.96inchOLEDModule 0.96inchOLED(A) 1.27inchRGBOLEDModule 1.3inchOLEDModule(C) 1.3inchOLED(A) 1.32inchOLEDModule 1.5inchRGBOLEDModule 1.5inchOLEDModule JETSONORINNX 自然语言处理 搜索引擎 VR 计算广告 深度学习 机器学习

    最新资料