Lab 7 – Robot Tournament Part 1

Prelab

Please read through the robot tournament introduction here and complete the tasks below.

  1. Review the slides from lab 5 (Connectionism) to refresh your memory of ML Agents
  2. Get together with your group before the lab and decide on a name for your robot. That will also be your group name for the tournament
  3. Have a chat about what you might want your robot to look like
  4. Play around with the demo (link also in the preliminary instructions) and think about:
    • What your agents need to be able to do
    • What strategies would work for this game
  5. Submit your answers for 2-4 on Canvas for lab 7 prelab

Link to the github code

Lab

Note that as of Nov 16 there is a new version of Unity available. DO NOT download this version. It’s fine to keep with the old one.

Part 1 – TA run through of the code/project

Your TA will take you through each component of the code we will be using for this project, as well as explaining the overall structure of the final project. You can find the slides we’re using here: Lab 7 Presentation – Cogs 300-1 2

  • This is a great time to ask any questions you have about the structure of ML agents or about the final project

Sections of the code (In order):

——- Basic Monobehaviour Functions ——

  1. Start
    • What happens at the beginning of the program.
  2. FixedUpdate
    • Will be called and run every frame

——- Agent Functions ————-

  1. CollectObservations
    • Inputs to your NN
  2. OnActionReceived
    • Translating the outputs from your NN into actions. Uses MovePlayer as helper.
    • This is where most of your code will be written for the final project
  3. Heuristic
    • Manual control which will simulate specific output from your NN

—— Override Functions ——–

  1. OnTriggerEnter
    • Called when you collide with an object that acts as a trigger. We use this to determine when you enter the area of a base. Triggers do not act as a solid object and you can pass through them.
  2. OnCollisionEnter
    • Called when you collide with all other objects. This is used for collisions with Walls and Targets and could also be used for detecting collisions with the other player .

—— Helpers ——

  1. MovePlayer
    • A helper function that is called from OnActionReceived. This is where we have implemented actual movement of the agent
    • This is where most of your code will be written for the final project
  2. GoToBase
    • Turns the agent in the direction of their own base and moves them towards it
  3. GoToNearestTarget
    • Turns agent in direction of nearest target and moves them towards it
  4. TurnAndGo
    • Turns agent in the direction of a target and then moveHelper for GoToBase and GoToNearestTarget.
  5. GetNearestTarget
    • Returns the target which is currently nearest to you
  6. GetYAngle
    • Tells you which direction to rotate to face a given object

Part 2 – Adding your group name to your robot

Your group name must not have spaces. Any numbers must have an “_” before them. 

  • Ex:
    • “TheBestRobot”
    • “The_Best_Robot”
    • “The_Best_Robot_3000”
  1. Open Scene “AgentTesting”
  2. Open folder Assets -> Resources -> MyGroupName
    1. Change the folder name to your group name
    2. Change the script “MyGroupName.cs” to your group name
    3. Open the script and change the name of the class to your group name
    4. Change the object to your group name 
  3. Open the script “WorldConstants.cs”
    1. Change the name of agent1ID and agent2ID to your group name
  4. Save your files
  5. Go back to unity and hit run
    1. You will only be able to move forward with the arrow key right now
    2. Make sure that there are no pink objects (means they haven’t loaded right) and that there are no errors in your console.

You DO NOT need to show your TA your work from this section, but you must get this working before you move on.

  • If you are stuck for more than 2 or 3 minutes on a stage in this section ask for help right away

Part 3 – Implementing OnActionReceived

  1. This section will take you through the process of adding basic movement to your agent. Finish implementing Heuristic, get your agent to move with keyboard. Look for “TODO-1” for the items in the code for this section (use ctrl/cmd + f to find them all). There is a spelling mistake on line 121 where it says “TODO”, but should be “TODO-1”
    1. In Heuristic(), follow the TODO-1 instructions
    2. In OnActionReceived(), follow the TODO-1 instructions
    3. In MovePlayer(), follow the TODO-1 instructions
    4. Go back to unity and try running it. You should now be able to move in all directions using the arrow keys.
  2. Lets pretend that we now want to add in an additional behaviour of going back to your base. Implement the go to base behaviour using the goToBase() function. The function is already created for you, but you must integrate it into your agent.  Look for “TODO-2” for the items in the code for this section. (use ctrl/cmd + f to find them all)
    1. In unity, open the inspector for your agent (Assets->Resources->[yourGroupName])
      1. Scroll down until you see the “Behaviour Parameters” component. Since we are adding in another type of behaviour to our agent, we must add an extra branch to our behaviour.
      2. Add the extra branch
        1. Add one to “Branches Size”
        2. In the newly created “Branch 4 Size”, set the size to 2
    2. Go back to the code to implement the TODO-2 instructions
      1. In Heuristic(), follow the TODO-2 instructions
      2. In OnActionReceived(), follow the TODO-2 instructions
      3. In MovePlayer(), follow the TODO-2’s for instructions
    3. Go back to Unity and try running it. You should now be able to move back to your base using the key you chose.

Show your TA your work from part 2 and 3. If you are waiting after use that time to work on your strategy for the game.

We will be moving on to part 4 as a group 30 minutes before the end of lab. Make sure you complete part 3 before that point.

Part 4 – Example training sequence (Time Permitting)

This section will involve a TA demo of how to conduct some basic training for your agent

Here is a link to some of the basic training commands which we will be discussing. Some of these should be familiar to you

Postlab

  1. Suggest two possible strategies for your robot in this game. Give a pro and a con for each strategy.
  2. Pick one of the strategies from question 1 and give a high level explanation on how you would implement it in the code (max 60 words). Note: we are not asking for pseudo-code, but rather a description in English e.g. which part of the code will be changed, what settings will need to be adjusted, what helpers will be added, etc.
  3. Tell us us any questions you have about the final project or ML Agents that you would like to be addressed in the next lab.