Wednesday, December 12, 2007

Apples to Oranges: Lessons in reward design from neuroscience?

According to news of paper by Padoa-Schioppa & Assad. in Nature Neuroscience, neurons in a primate's orbitofrontal cortex correlate to behavioral preference for a juice when presented with a ratio of apple and grape juice. Moreover, the neural correlation was invariant regardless of other options, suggesting that preference transitivity (A to B, B to C, therefore A to C) is reflected in neural activity of the orbitofrontal cortex. As commenter Neil Farbstein wrote, it's "comparing apples to organges!"

This research was performed on rhesus monkeys. I wonder what their preference was in terms of a ratio of duration of how many days in testing 600 to 2400 trials of sips of juice compared to how many days surviving in the wild. Their methods on one male and female rhesus monkey were:

Under general anesthesia, we implanted a head-restraining device and a recording chamber on the skull of the monkeys, and implanted a scleral eye coil. We used large, oval, custom-made chambers (main axes 50 x 30 mm), centered on stereotaxic coordinates (A30, L0), with the longer axis parallel to a coronal plane. Following surgery, monkeys were given antibiotics (cefazolin, 20 mg per kg of body weight) and analgesics (buprenorphine, 0.005 mg per kg; flunixin, 1 mg per kg) for 3 d. During the experiments, monkeys sat in a monkey chair in a darkened room. The head was restrained and the eye position was monitored continuously using a scleral eye-coil system (Riverbend Instruments). A computer monitor was placed 57 cm in front of the monkeys, and the behavioral task was controlled by custom-written software.

The actual interface of the simulation was a trivial computer game (or computer investing) of sorts: An offer appears on screen as money and delay (like "$40 / 30 days"), a button is held down, six seconds pass, a green dot appears, and the subject releases the button. For motor task independence, half the trials release indicated immediate ($20) or delayed reward (variable). The amount on the screen was money that would be given after the duration of delay.

The article recounted the hyperbolic function of subjective value: SV = 1 / (1 + kD); in which subjective value (SV) equals inverse of subject-specific constant (k from 0.0005 to 0.1189) and delay of reward (D in days). This function might have application to reward design in games, especially if a similar formula holds true on the time scale of seconds or minutes of a single playing session. An example exists in harvesting or saving the little girls in BioShock. Harvesting has an immediate reward whereas saving offers a delayed reward. Are there other examples that illustrate delayed reward design?

Especially for multiplayer games, the relative ranking of the reward compared to others is important, at least for men. Experimentin neuroeconomics lab at University of Bonn says the activitation of neural reward evaluators in the brain correlate to getting a higher reward or performing better than another participant. So competition, is rewarding for the winners to the extent that there are losers. And advance in a society is measured by surpassing the Joneses.

Comments (2):

Gayle wrote




Gayle wrote


Ethan, I wonder if the "head-restraining device and a recording chamber" on the skull of the monkeys looks something like this? http://www.novivisezione.org/mostra/pan3_en.htm