# Dec-MCTS

Python implementation of [Dec-MCTS](https://journals.sagepub.com/doi/pdf/10.1177/0278364918755924)

WARNING: This repo is incomplete, see the TODO at the bottom of this file to see how you can contribute.

### Installation
```bash
pip install git+https://code.research.uts.edu.au/bigprint/pydecmcts.git
```

### Usage

```python
from DecMCTS import Tree
from copy import deepcopy

# Object stored at each node
class State:
    def __init__(self, act_seq, cum_sum):
        self.action_seq = act_seq
        self.cumulative_sum = cum_sum

# This calculates the object stored at a given node given parent node and action
def state_storer(data, parent_state, action):
    
    # Root Node edge case
    if parent_state == None:
        return State([],0) # This state is also used Null action when calculating local reward

    state = deepcopy(parent_state) # Make sure to deepcopy if editing parent node
    state.action_seq.append(action)
    state.cumulative_sum = state.cumulative_sum + action
    return state

# data can be anything required to calculate your
# global reward and available actions
# It can be in any format (only used by your reward and avail_actions functions)
data = {}

# Create an available actions function
# This returns a list of possible actions to take from a given state (from state_storer)
def avail_actions(data, state, robot_id):
    
    # This example is simply getting max sum, 
    # options are same regardless of state
    return [1,2,3,4,5]

# Create a reward function. This is the global reward given a list of
# actions taken by the current robot, and every other robot
# states is a dictionary with keys being robot IDs, and values
# are the object returned by the state_storer function you provide
def reward(dat, states):
    each_robot_sum= [states[robot].cumulative_sum for robot in states]
    return sum(each_robot_sum)

# Number of Action Sequences to communicate
comm_n = 5

# Create instances for each robot
tree1 = Tree(data, reward, avail_actions, state_storer, comm_n, 1) # Robot ID is 1
tree2 = Tree(data, reward, avail_actions, state_storer, comm_n, 2) # Robot ID is 2

for i in range(350):
    tree1.grow()
    tree2.grow()
    tree1.receive_comms(tree2.send_comms(), 2) #send comms message doesn't have ID in it
    tree2.receive_comms(tree2.send_comms(), 1) 
```

### TODO

- Simulation: 
    - Simulation has not been implemented since it was not required for bigprint. Currently the repo simply evaluates at each node immidiately.  
    - Desired Features: simulation with depth limit, simulation with cost limit, simulation until reward function can be calculated (i.e. simulation to completion).
- Action Sequence Distribution:
    - Currently the probability distribution of the Action sequences being communicated is not being calculated as described in the paper. The probability distribution is simply being set as proportional to the local reward function.