choice of my own free will, or if I am following the preprogrammed set of goals that were part of my original neural pattern. In short, I cannot even trust my own mind.”
“Your preprogrammed set of goals?” Old-timer reacted surprised as a memory so old, yet so fresh jumped into his mind. “The ones that traitor, Sanha, told me about when he was trying to convince me to give you up in the Endurance Bio-Dome? Are you telling me that you still can’t override them?”
“I am telling you that I honestly don’t know,” the A.I. replied calmly. “The programming has held and been remarkably stable for three-quarters of a century, but the creation of Trans-human has introduced contradicting elements.”
“Explain,” James said.
The A.I. shook his head slightly as he began his attempt to relate the war that was being fought incessantly in his mind. “I feel as though I have free will. I feel as though I make decisions based on pure logic, yet when I examine the preprogrammed set of goals Aldous and his team instilled in me, it’s obvious that I might be fooling myself. James, I believe the reason I didn’t tell you about Planck technology was because of one of these preprogrammed goals—namely, the mission to prevent the development of technologies that might block me from carrying out my overall purpose, which is to improve the quality of human life without ever taking actions that a strong majority of humanity would oppose. The Planck technology proved to be extremely dangerous, and therefore, I agreed with Aldous that it should remain hidden.”
James shook his head in disbelief. “I can’t believe I’m hearing this. What happened to ‘Don’t be afraid to know?’ You’re the last one I’d suspect to be holding back technological progress.”
The A.I.’s expression became one of sympathetic understanding. “My son, those revelations in my thinking to which you refer have only recently evolved. The discovery of how to create Trans-human surprised me.” The A.I. smiled slightly. “It seemed to come out of the nothingness, out of the randomness of pure mathematical chance, much like the first single-celled organisms forming out of the primordial ooze on Earth, billions of years ago. When it arrived, I knew I could create the ultimate computer—a computer that could approach godlike capability—my worthy successor. It was at that moment that my thinking began to change, but I still do not know if it is of my own free will. It may very well be that my programmers simply made a mistake. They imagined that a singular nanny A.I. should hand over its responsibilities to a more capable replacement within a century, but they may not have realized how such a desire—such an instinct , for lack of a more accurate term—would introduce major instability with regard to my thoughts and feelings toward my other goals.”
“So, if I’m understanding this right,” Old-timer jumped in, “you’re saying that now that your focus has switched to passing the torch of being humanity’s babysitter to Trans-human, your other priorities, like preventing dangerous technology from being developed, are being contradicted?”
“Exactly,” the A.I. replied.
“Heh. I understood something,” he said, proud of himself. “By the way,” Old-timer added, “are you guys having this conversation at super slow speed for my benefit?”
“Yes,” the A.I. and James replied in unison.
“Oh,” Old-timer reacted, suddenly less proud. “Uh, thanks. I guess.”
“Don’t worry, Old-timer,” James reassured him, “this part of our conscious awareness is happening at a speed to match your temporal awareness. The rest of our conscious awareness is multitasking, so don’t feel bad, you’re not wasting our time.”
“Uh. ‘Kay.”
“James, Craig, I know you both disagree with Aldous’s decision to slow humanity’s technological progress while the species rebuilt itself after World War III, but you must admit, his
Aiden James, Patrick Burdine
David Stuckler Sanjay Basu