MyNixOS website logo
Description

Interface to 'pomdp-solve' for Partially Observable Markov Decision Processes.

Installs an updated version of 'pomdp-solve', a program to solve Partially Observable Markov Decision Processes (POMDPs) using a variety of exact and approximate value iteration algorithms. A convenient R infrastructure is provided in the separate package pomdp. Kaelbling, Littman and Cassandra (1998) <doi:10.1016/S0004-3702(98)00023-X>.

R package pomdpSolve - Interface to ‘pomdp-solve’ for Partially Observable Markov Decision Processes

CRANversion stream r-universestatus CRAN RStudio mirrordownloads

Provides pomdp-solve (Cassandra, 2015) to solve POMDPs using a variety of algorithms. A convenient interface is provided in package pomdp.

Installation

Stable CRAN version: Install from within R with

install.packages("pomdpSolve")

Current development version: Install from r-universe.

install.packages("pomdpSolve", repos = "https://mhahsler.r-universe.dev")

Usage

Solve a POMDP file that ships with this package in a temporary directory and read the solution.

library(pomdpSolve)

old_wd <- setwd(tempdir())
file.copy(system.file("tiger.aaai.POMDP", package = "pomdpSolve"), "./tiger.aaai.POMDP")
## [1] TRUE
pomdp_solve("tiger.aaai.POMDP", options = list(method = "grid"), verbose = FALSE)
dir()
## [1] "tiger.aaai-0.alpha" "tiger.aaai-0.pg"    "tiger.aaai.POMDP"
read_alpha_file("tiger.aaai-0.alpha")
##         V1    V2
## [1,] -98.5  11.5
## [2,] -10.9   6.5
## [3,]   1.9   1.9
## [4,]   6.5 -10.9
## [5,]  11.5 -98.5
read_pg_file("tiger.aaai-0.pg")
##   N A Z1 Z2
## 1 1 2  3  3
## 2 2 1  3  1
## 3 3 1  4  2
## 4 4 1  5  3
## 5 5 3  3  3
setwd(old_wd)

The value function is defined by a set of alpha vectors represented as rows in a matrix with $V$ components. Each alpha vector corresponds to a node with the same row number in the policy graph. The policy graph represents the policy in the form of a data.frame with one row for each node. Nodes have a node id $N$, the action $A$ defined by the policy and then for each observation $Z_i$ the id (row) of the policy graph node to transition to.

A detailed description of the available solver options can be found here.

How to Cite this Package

To cite package ‘pomdpSolve’ in publications use:

Hahsler M, Cassandra AR (2023). pomdpSolve: Interface to ‘pomdp-solve’ for Partially Observable Markov Decision Processes. R package version 1.0.3, https://github.com/mhahsler/pomdpSolve.

A BibTeX entry for LaTeX users is

@Manual{, title = {pomdpSolve: Interface to ‘pomdp-solve’ for Partially Observable Markov Decision Processes}, author = {Michael Hahsler and Anthony R. Cassandra}, year = {2023}, note = {R package version 1.0.3}, url = {https://github.com/mhahsler/pomdpSolve}, }

References

Metadata

Version

1.0.4

License

Unknown

Platforms (75)

    Darwin
    FreeBSD
    Genode
    GHCJS
    Linux
    MMIXware
    NetBSD
    none
    OpenBSD
    Redox
    Solaris
    WASI
    Windows
Show all
  • aarch64-darwin
  • aarch64-genode
  • aarch64-linux
  • aarch64-netbsd
  • aarch64-none
  • aarch64_be-none
  • arm-none
  • armv5tel-linux
  • armv6l-linux
  • armv6l-netbsd
  • armv6l-none
  • armv7a-darwin
  • armv7a-linux
  • armv7a-netbsd
  • armv7l-linux
  • armv7l-netbsd
  • avr-none
  • i686-cygwin
  • i686-darwin
  • i686-freebsd
  • i686-genode
  • i686-linux
  • i686-netbsd
  • i686-none
  • i686-openbsd
  • i686-windows
  • javascript-ghcjs
  • loongarch64-linux
  • m68k-linux
  • m68k-netbsd
  • m68k-none
  • microblaze-linux
  • microblaze-none
  • microblazeel-linux
  • microblazeel-none
  • mips-linux
  • mips-none
  • mips64-linux
  • mips64-none
  • mips64el-linux
  • mipsel-linux
  • mipsel-netbsd
  • mmix-mmixware
  • msp430-none
  • or1k-none
  • powerpc-netbsd
  • powerpc-none
  • powerpc64-linux
  • powerpc64le-linux
  • powerpcle-none
  • riscv32-linux
  • riscv32-netbsd
  • riscv32-none
  • riscv64-linux
  • riscv64-netbsd
  • riscv64-none
  • rx-none
  • s390-linux
  • s390-none
  • s390x-linux
  • s390x-none
  • vc4-none
  • wasm32-wasi
  • wasm64-wasi
  • x86_64-cygwin
  • x86_64-darwin
  • x86_64-freebsd
  • x86_64-genode
  • x86_64-linux
  • x86_64-netbsd
  • x86_64-none
  • x86_64-openbsd
  • x86_64-redox
  • x86_64-solaris
  • x86_64-windows