The researchers had agents play 50 games using different RL methods. They found that methods utilizing the cross-entropy policies performed better than methods that were hand-crafted. As they explained, the basic idea of cross-entropy is that it selects the most successful actions, and modifies the distribution of actions to become more peaked around these selected actions.
During the game, the AI agent must make decisions on which way to go, which are governed by ruled-based policies. When the agent has to make a decision, she checks her rule list, starting with the rules with highest priority. In Ms. Pac-Man, ghost avoidance has the highest priority because ghosts will eat her. The next rule say that if there is an edible ghost on the board, then the agent should chase it, because eating ghosts results in the highest points.
One rule that the researchers found to be surprisingly effective was the rule that the agent should not turn back, if all directions are equally good. This rule prevents Ms. Pac-Man from traveling over paths where the dots have already been eaten, resulting in no points.
Commodore’s C64 had a famously decisive, if drab set of 16 colors to choose from, a note of artistic intent amid the unthinking mathematical extremities of other 8-bit color palettes. But did you know there were secret colors? Aaron Bell writes up a discovery that blew his mind many years ago and which, 26 years […]
The Offworld Collection, presenting the very best features and essays from Offworld, is finally available to buy directly from Campo Santo for $40. I had the pleasure of designing and illustrating this splendid 250-page hardcover volume, but it’s the excellent writing, edited by Leigh Alexander and Laura Hudson, that makes it an essential buy. You […]
The Lightning port has thus far resisted the cruel fate that befell the headphone jack, and despite rumors that it may be disappearing come iPhone 8, for the present and foreseeable future, Lightning cables are a hot commodity for iPhone users. As such, we must make do in this strange time in which long, glorified […]
All the filters in the world won’t save your smartphone pics from a shaky hand. To really step up your mobile photography game, you’ll need some kind of mount to hold it steady. You could buy a smartphone attachment for a conventional camera tripod, but who wants to carry that kind of gear everywhere they […]
The forced transition from analog to digital TV signals was probably met with relative indifference from people with Netflix subscriptions and the “I don’t even own a TV” snoots. But anyone living in the vast swaths of the country that don’t have guaranteed high-speed internet, broadcast TV is a perfectly valid (and 100% free) way […]