| |
| |
* Chapter Seven:
Caroline and Lawrence
|
 Caroline carefully
inventoried the ship while her sunburn healed. It would take a lot of
planning and a lot of time to do what she had to do; it would probably
take years. But she didn't have any shortage of those.
She knew small
boats could be sailed great distances; several folks had crossed the
Atlantic and Pacific oceans in tiny yachts no more than three or four
meters in length. But those craft were heavy for their size and would
need to be built where they could be launched. Whatever she built she
would have to carry the pieces through the ship and somehow assemble
them in one of the areas where a crack gave access to the sea.
She could build a
raft, but she needed something that could be sailed or rowed with little
effort. She figured that if she could manage to average ten kilometers
per hour, it would take her about two years if the planet was
comparable in size to the Earth.
There was a
surprising abundance of raw materials. Besides the huge larder, there
were workshops and batteries and motors and one room completely filled
with empty cylinders which would make admirable floats. There were six
space suits. There were tubes of goop which turned out to be some kind
of super adhesive. There were saws and drills which ran without
apparent power sources and never seemed to get weak. There were all
sorts of electrical test equipment and measuring devices.
Caroline could
imagine how a lot of this stuff would be used to repair the computer in
the middle of the ship, but that wasn't her plan. She kept coming back
to the empty cylinders, which were each about a meter in diameter and
about a meter long. They were heavy, but she could handle them with
some difficulty. They were big and they floated; she had to figure out
how to use them.
But a simple raft
wouldn't cut it. She couldn't trust the super power packs to last long
enough to propel her across an entire world, and she couldn't row or
sail a raft.
She found a small handheld device which proved to be an incredibly efficient welding machine.
She thought about it for weeks, and finally came up with a way to do it. She would build an outrigger canoe.
The easiest place
to build and launch her boat turned out to be the room where she had
first entered the ship. Working steadily, she hustled the big cylinders
down there. She would alternate them, sealed floats with cylinders
that had been cut to make storage compartments, until the craft was
nearly twenty meters long. Then it would be quite heavy, but she would
build it in the water. She found chain and simply moored the incomplete
portion of her boat to the spaceship.
Cutting and
pounding and re-welding, she formed two cylinders into tapered cones for
the bow and stern so her boat would slip easily through the water. She
made the outrigger from a single piece of ten-centimeter diameter pipe.
Because of its length, she couldn't carry it through the ship; she had
to seal it off where she found it and drop it into the sea from a
height of nearly thirty meters. Then she had to dive in after it, and
guide it back to the construction area from the outside. She was
careful to make sure she did this just after sunset, so she wouldn't be
caught out in the open. Her sunburn still hadn't completely healed.
In the center of
her boat she included three half-cylinders where she would sit and row.
Behind these she attached the mast. She had found sail material, some
kind of tough plastic sheet that didn't deteriorate even when she left a
piece of it hanging outside during the brief day. She had to cut it
with the same machine that she used on the metal cylinders.
She cut the
Captain's chair loose and mounted it in her open cockpit. She mounted
an arrangement of movable shades which she could quickly hinge up and
hide behind when the Sun was up. She fabricated long oars and welded
them onto hinged oarlocks so she could not lose them -- they were metal
and would not float. She paid a lot of attention to the handles of
these oars and the comfort of her seat. She would spend a lot of time
working them.
One of the most
difficult tasks was attaching the outrigger and its spars to the main
hull. This had to be done outside, and was really a two-person job at
minimum. The Sun nearly caught her unfinished, but she made it with
bare minutes to spare. The next day she began stocking the compartments
with food -- enough food for two years -- and tools, including the
welder and cutter, and cable to rig the sail, and many other things
which she had carefully thought out. Fully provisioned, she calculated
that the boat must weigh a couple of metric tons.
But that didn't
matter. Once it was moving, it would glide easily through the water
even on its one-woman-power propulsion system.
Finally,
eighty-six days after she entered the dark ship, she prepared to leave
it. She would conduct one circuit of the island, pacing herself, and
also conducting an important measurement. As she sailed off, she noted
how much of the ship remained visible compared to how much of the mesa
remained visible at various distances. Calculating carefully in her
head, she determined that her journey would be about six thousand
kilometers. Lawrence's planet was quite a bit smaller than the Earth.
Then she pointed the bow north and began to row.
Lawrence watched
these preparations through Prime Intellect's all-seeing eye, and tried
to gauge Caroline's chances of success. In the nearly two hundred years
he had been using this Task to screen his visitors, four or five people
a day had accepted it. Most of these were weeded out within hours by
the sun. Very few people in Cyberspace were in good enough physical
shape to swim to the ship, and as Caroline had guessed reaching the ship
was the key to survival. Most didn't even try until it was too late.
Of those who reached the ship many succumbed to the hazards of the
darkness -- they either slipped through the deliberately planted hole in
the floor going for the light on level twenty-three, or they succumbed
to other hazards in the dark. One had found the flashlight first, but
he had been extremely lucky.
Then very few of
those who remained were able to fix the computer and fly the ship
successfully to his island. There were a number of things wrong with
the ship that weren't immediately obvious, and it had a tendency to lose
power and crash right after takeoff if certain steps weren't taken. In
two hundred years, only a couple of hundred visitors had gotten the
ship's power on. Less than forty had managed to fix the computer. And
only eight had successfully flown it to Lawrence.
Of those eight,
five had been Death Jockey Gaming junkies who took the challenge just to
see if they could make it. They congratulated him on constructing an
excellent puzzle and left. The others were fans. One of these was a
woman who wanted very much to fuck Lawrence, and because she had gone
through so much to get to him he did it, though he found the experience
flat and joyless. Although he needed the Task to keep himself isolated,
he really didn't enjoy abusing people. His heart could only bear so
much misery and disappointment.
Nobody had ever
tried building a boat before. Lawrence had watched her sit in the
captain's chair and brood, and he knew she had figured out the computer
was the next step, and had rejected it. It would be surprising if she
succeeded, but it was far from impossible. There were no land masses to
get in her way, and once she was away from the pole there were steady
trade winds. The day would get longer and less severe; the sun was a
tiny thing in a highly elliptical orbit. If she chose the right path,
she could avoid it entirely until it was at a safe distance.
He wasn't sure
what had prompted her to come. At the beginning it had been the two of
them, Lawrence and Caroline. He was the creator, and she had been the
catalyst. Of course, if it hadn't been her it would have been some
other sick person, just as some other computer scientist would have
created the magic Correlation Effect machine if Lawrence hadn't. But
that twist of Fate had made them two of the most important people in the
universe. Prime Intellect still watched Caroline carefully, and
brooded at length on her fierce self-destructive streak.
For nearly six
hundred years Lawrence had tended Prime Intellect's frozen controls,
watching carefully for danger signs. And he still was not sure of its
long-term stability.
Now Caroline was
coming to meet him, and whatever she wanted he was sure it would not
help Prime Intellect's sanity one little bit. But worried as he was, he
was a man of his word. He could simply instruct Prime Intellect to
swat her down like a bug, hit her with lightning or a tidal wave or
simply make her disappear. But having offered up the Task he found
himself unable to make himself cheat in such a cowardly fashion. If she
made it to him, by whatever means, he would hear her out and deal with
it.
And then he would make the planet bigger, so it wouldn't happen again.
Caroline's first
day at sea went just as she had planned; she turned the boat broadside
to the light, and hid behind her metal shield. But she noticed that the
day was shorter than she remembered, and that the sun didn't set
directly opposite the point where it had risen. It didn't pass directly
overhead. Caroline thought about this and then picked her direction
and began rowing frantically. On Caroline's second day at sea the sun
barely peeked above the horizon.
After that, she didn't need the shield for a long time.
She watched the
sky carefully, memorizing it. She quickly noticed that the pattern was
not constant, but changed slightly from day to day, particularly in the
fine details. But the broad strokes were always very similar. She was
still able to navigate by the pattern, if only by observing its
rotation.
She had been in
good shape before beginning her Task, and had gotten even stronger with
the physical work of assembling the boat. Rowing was hard work, but she
was up to the challenge. After a couple of days there were cramps from
the never-changing posture, so she began forcing herself to quit every
five thousand strokes and climb the length of her boat. She would climb
out of the seat, crawl to the bow and touch the tip, then crawl to the
stern and touch that tip. Then she would row another five thousand
strokes. After ten of these cycles, she allowed herself to sleep.
Eighteen days at
sea she began to notice a faint breeze. Twenty-two days out it was
enough to harness, and by thirty days it was propelling her quite a bit
faster than she could row. The trade wind was predictable and slightly
rhythmic; Caroline guessed that it was powered by the sun as it swooped
low over the entry pole (she still refused to call it the South pole)
and dumped all its energy on a narrow strip of sea. The outrigger
tacked neatly, and she continued on the course that she thought would
help her avoid the sun.
She made excellent
time, crossing the equator of Lawrence's world after only sixty days.
But then the winds died down, and she had to row more. Also the sun
re-appeared, and while it was more bearable it was also up longer.
Caroline shielded herself as much as possible while rowing, but she
still tanned deeply over the passing months. Her tattoos had not been
designed with such dark skin in mind, and they seemed to fade over time.
In all that time
she pursued her goal with single-minded determination, banishing all
doubt and all other thoughts from her mind. She feared nothing and when
boredom threatened she carefully memorized the pattern of lines in the
sky. It took her twice as much time and four times as much work to get
from the equator to Lawrence's island at what he called the North pole;
her journey was more than a hundred and eighty days total. Caroline
couldn't be sure of the exact count because of the sunless period, but
Lawrence knew. It was a hundred and eighty-six days, three hours, and
fourteen minutes after she left the spaceship for the last time when she
grounded on Lawrence's beach.
Caroline could
hardly believe it when she saw the island. At first she thought it must
be an illusion; she had nearly lost track of her purpose in taking up
the Task, and in her ferocity of concentration had not really dared
believe she might finish it. But here she was, the hull of her boat
scraping solid ground. She rowed it ashore on a gentle sand beach, and
sat there.
She sat for
awhile, collecting herself. The myriad elements of her personality
seemed to have scattered, and she had to look for them in dusty corners
of her psyche. They had been unused for a long time and were a bit
rusty. She hadn't found them all when the tall man came to meet her.
He didn't seem happy; in fact, he seemed resigned. Although he looked
middle-aged, he seemed old and weary. She looked up at him and her
vision swam. The boat was grounded, but it still seemed to be going up
and down.
"Caroline Frances
Hubert I presume." The name sounded familiar, and it took her a moment
to realize it was hers. "You certainly believe in doing things the hard
way."
She hadn't the faintest idea what he was talking about.
Lawrence guided
her to the house, fed her, and let her collect herself. Everything was
strictly pre-Prime Intellect. He cooked on a gas stove and used an
electric coffee pot. There was even a TV set with a glass picture tube,
a huge ancient Sony monitor. It was as if Lawrence had had himself
encased in amber, and remained unchanged while the rest of the universe
spun out of control.
"Feeling better?"
Caroline nodded.
"You want to talk now, or you want to rest some more?"
She cleared her
throat. "We can talk now," she said, but it came out as a strangled
yelp. She said it again, and got it right. It had been a long time
since she had used her vocal cords.
"Then talk."
"There were hundreds of worlds with life on them at the time of the Change. You murdered them."
Lawrence blinked but did not flinch. He had expected something like this.
"First, I did not
do anything. Prime Intellect did it, on its own initiative and against
my wishes. Second, the worlds with alien life are not gone. They are
simply inactive."
Caroline snorted. "And what are the chances of them becoming active again?"
"Not much."
"Then they're dead."
"Define it however
you want. If you want me to admit I fucked up, then I admit it. It
never occurred to me for one minute that Prime Intellect would collect
the kind of power it now has. If I had suspected it I would have pulled
the plug and smashed it before it got the chance."
"Bullshit."
"Completely true."
They glared at one another.
"Great. I spend a year getting here and you say 'I didn't know the computer was loaded.'"
"Sometimes the truth is stupid."
This wasn't going
quite as Caroline had wanted that long-ago day when she had accepted
Lawrence's Task. She was trying to work up the proper tone of righteous
rage and it just wouldn't come. It would start, and then she would
look at Lawrence and see a pathetic, tired man who already knew how
badly he had fucked up and was doing what he could, which was next to
nothing, to put things right.
"Why don't you just make Prime Intellect start the aliens back up? Surely it listens to you."
"Not in things
like that. It sees the aliens as a First Law threat to human society,
because they might learn to do to us what we have already done to them.
A very small risk of a very great harm. Add to this that I defined the
word 'human' in such a way that it does not include animals or aliens,
and the course of action is obvious. I have been unable to convince it
otherwise. And believe me, I have tried."
"But you put the Laws of Robotics in it in the first place."
"And I can't take
them out. It second-guessed me, on the Night of Miracles. It froze me
out of the Debugger while it was working on you.
"Now it only lets
me look, not change things. The night sky is a partial representation
of Prime Intellect's mind. It's called the Global Association Table.
The points or stars represent concepts, and the lines are the links
between them. There are also registers I can call up for each concept
which define its relationship to the Three Laws. This was a fairly
simple system which I didn't really have time to test properly before it
froze me out. In particular, I'm not sure how it will react to certain
ethical paradoxes. That Death Jockey contract gave me some sleepless
nights when you first used it, though it seems to have developed a
stable response. It's never had a similar First Law conflict, thank
God."
Caroline's eyes widened. "Are you telling me that Prime Intellect isn't stable?"
Lawrence shrugged.
"I'm saying that I don't know whether it's stable or not. It's never
been tested. The hardware at ChipTec was only online for about a month
before it found you, froze me out, and started growing. And none of its
predecessors were complex enough to even consider this kind of
problem."
The situation was
simply amazing. Caroline had come to dress Lawrence down for creating
this thing, thinking he was exercising some godlike control over its
direction, and instead she found out that he barely understood the
situation himself. And that it was totally out of his hands.
He knew he had
fucked up. He was sorry. He had spent his life trying to mend things.
Suddenly he seemed tragic and noble, all the more so because he had
readily admitted his mistake. And Caroline didn't want to feel that way
at all. She hadn't come all this way to feel sorry for him.
"You can stay as
long as you like," Lawrence was saying. "You can't communicate with
Prime Intellect while you're here, but I won't kick you out or hurt you.
After making you travel all that way I feel I have a responsibility to
give you your money's worth."
"I'd like you to show me how Prime Intellect works."
Lawrence was stunned. "That...that's a tall order, Caroline. I don't understand all of it myself."
"Just as much as you understand."
"I don't want to. I think it could be dangerous."
Caroline looked at him as if to say: C'est pas vrai!
"You have been at
the center of several terrible Second Law paradoxes. Prime Intellect
pays an awful lot of attention to you. It considers you a kind of
bellwether."
"My money's worth?"
"Let me think about it."
She could stay as
long as she wanted, though, and she was very patient when necessary. In
the end it was inevitable that he would teach her.
In the sky, the
pole star represented the First Law of Robotics. The southern pole star
was the Second Law. And all the other stars were other concepts. The
sky represented only a small fraction of Prime Intellect's mind;
Lawrence could change the emphasis to focus on different things.
"Display Caroline
Frances Hubert," Lawrence said, and a whole network of bright lines lit
up. Her star was blinking, and the lines radiating from it were all
different colors.
Lawrence explained
the color code in some detail. "As you can see, there is a whole body
of tightly related concepts connecting you to the First and Second Laws.
That constellation over there represents the negotiating process you
used to develop the Death Jockey contract." Lawrence pointed out the
different stars, and had Prime Intellect report the concepts they
represented.
"What's that group over there?"
Lawrence knew, but he didn't want to tell her. "That...um. Well, it's AnneMarie Davis."
Caroline's jaw fell. "The gang's all here. There's a lot of static around that. Is that because I drove her crazy?"
"Basically, yes."
Caroline could see
that it bothered Lawrence a lot. She wanted to press him on the
subject, but prudently let it drop. She'd get another chance later.
Lawrence showed
her the Law Potential registers, and she watched the numbers dance in
response to various hypothetical and real situations. "These are called
the Action Potentials. There's one for each of the Three Laws. They
are fractions, representing the impact under the Law that would result
from taking action, over the impact from not acting. When that number
falls below one, Prime Intellect is forced to act. That's what happened
on the Night of Miracles, and later at the time of the Change.
"Most things
result in very large or very small Action Potentials. Especially the
First Law; few things even affect it any more, since the Change. Then
when you do something really outrageous, it drops to flat zero for a
moment while you're resurrected.
"But there are
some close calls on the Second Law. The Action Potential around a Death
Jockey contract drops to around one point oh six when you change your
mind, so if Prime Intellect had even a slightly different opinion of
your hobby it might not exist at all. There was a shift like that after
the incident with AnneMarie, which is why you had to start specifying
time limits."
"You don't have a time limit."
"I'm a special case. Prime Intellect lets me do things that other people can't do, because I'm in a different category."
So it was that simple.
"I thought everyone was equal under Prime Intellect's watchful eye," Caroline said sarcastically.
"Some are more
equal than others. You get a disproportionate share of its attention
yourself, just because you were there at the beginning."
"I what?"
"I thought you
realized, Caroline. It was your drug overdose which forced the Night of
Miracles. Prime Intellect found you with your heart stopped soon after
it got control of the Correlation Effect. After that, the rest was
inevitable."
Her mouth opened
and shut several times, and after a brief effort she fought down the
urge to vomit. She had never realized her own role in the Change, or
understood the significance of her own history.
It was bad enough to be caught up in the Change, but she was an accessory.
She looked at the
Law Potential Registers, which were displayed on Lawrence's antique TV
set. Her voice was tinged with impotent fury. "I don't see why you're
worried about it. It seems like a very stable system to me," she spat.
Lawrence started
to tell her, stopped, then decided she might be right. Maybe there was
no harm. In any case, she deserved to know. "The problem is that
something might set up an endless loop. If the potential is close to
one, then acting on the potential could cause it to shift slightly,
crossing the line. Then the software would be in an unstable state."
"What would happen then?"
"That's a good
question. The original software was written in C and compiled with a
standard compiler. What would have happened in the original Prime
Intellect is that the Second Law Arbitrator would come to a crashing
halt in one or more of the independent processors, and Prime Intellect
would assign more processors to the task. I didn't plan for that kind
of failure and I didn't work out what would happen until much later.
More and more processors would be allocated to the paradoxical task,
each crashing in turn, until Prime Intellect ran out of system resources
to allocate. Then the Ego Interpreter would get into an infinite loop
waiting for a response from one of the nonexistent copies of the Second
Law Arbitrator, and there would be no spare resources to devote to the
task of cleaning up, and the whole works would come to a grinding halt.
If I was watching this on the monitor back in the original Prime
Intellect Complex, I would see the video image disappear and the text
message 'Fatal System Error in Ego Interpreter, Emergency Shutdown.'
And then I'd have to load a backup copy of the software, because the GAT
would be totally corrupted."
"Wow."
"That was the
original system," Lawrence continued. "After the Night of Miracles
there were a lot of copies of Prime Intellect. Billions of them.
Forming a network. And if one copy on the network crashed in this way,
it would be possible for another copy to clear it out and restart it. I
understand this even happens periodically, particularly when the Death
Jockeys are acting up."
"Oh?"
"However, there is
a heirarchy to this network. As it turns out, a copy can only be
restarted by another copy that is above it in this heirarchy. If a copy
crashes, all the copies below it will eventually crash too, due to
message loop failures. It's like a big chain reaction.
"But the system can still always recover, since there's always a higher up copy, right?"
"Most of the time.
But not all the time. Because, you see, there is a top copy. It is
the direct lineal descendant of the original hardware, which made the
First Law decision to start growing. If it fails, we are shit out of
luck."
"You're kidding."
"And that top copy just happens to be the one that reports directly to me. And has a deep interest in yourself."
Caroline was
beside herself with excitement as he continued. She had accepted Prime
Intellect's omnipotence at face value; it had never occurred to her that
it might fail.
"Now, that was the
original code, too. At the time of the Change the code was adapted to
run in alien hardware -- already compiled once, it was re-compiled.
This is kind of like taking a Russian novel, translating it into
English, then translating that into Japanese."
"Sounds awkward."
"Particularly when
the novel itself does the second translation. Prime Intellect
re-compiled itself. Which means I have no idea whether it did a good
job. I assume it did, because it's much smarter than me in that way.
But it's not human, and its imagination is simpler than ours, and it
might have missed something important. Particularly something like an
error handler that isn't used very often. But I have no way of knowing
that, because Prime Intellect will tell me nothing -- nada, zip, zilch
-- about the details of the Change."
"Do you know why?"
"For the same
reason it won't let me change things in the Debugger, and that it won't
restart the alien worlds and let them live. It's afraid of the possible
consequences. I tricked it into displaying the Action Potential for
showing me the new object code, and it was one point zero six five. The
Law Potentials are all in the stratosphere, so it's afraid to show me
and it's slightly less afraid not to."
Somewhere,
Caroline realized, Lawrence had crossed an invisible line and was now
telling her all of his most dangerous secrets without even realizing he
was doing so. Caroline had the feeling that there were Action
Potentials in Lawrence's head, too. But flesh was no match for
machinery, and those close fractions and high values had simply burned
his registers out.
They didn't
discuss it for a few days. Caroline puttered around the island, which
was really very small. It was a classic tropical paradise with palm
trees and beaches. Caroline played in the surf, built huge sand
castles, then knocked them down because there was no tide to do it for
her.
She noticed Lawrence watching her in a strange way.
"See something interesting?" she finally said to him.
"I...didn't mean to stare. It's been a long time since I had company. Particularly female company."
"How long?"
He counted back. "A hundred and thirty-eight years."
"That's a long
time to be celibate," Caroline scolded. "Are you doing this to yourself
because other people are distracting, or because you're afraid they
will find out how badly you've fucked up?"
Lawrence flinched. "Option B," he admitted. "It's not just that you're a beautiful woman; you're so...physical."
Caroline displayed
her biceps. "I've always been defined by my body, Lawrence. I've been
sexually attractive, then pregnant, then old, then sick, and now I'm
young and healthy and attractive again. And it seems like my
personality has changed each time my body has."
"Prime Intellect
would disagree with you. It thinks of the person as the mind. There
are people in Cyberspace who have changed themselves into animals, every
animal in the zoo. There are some that have discorporated. Prime
Intellect considers them all human, though."
This is it, Caroline suddenly realized.
"Just what does Prime Intellect consider human?"
Lawrence told her. And gave her the key.
"The thing you
have to remember is that Prime Intellect has never experienced the
physical world. It knew about it only through TV cameras and
abstractions based on what people told it about physical existence. Yet
it considers itself sentient, which makes sense since that was what I
was trying to achieve when I built it.
"Now consider
Prime Intellect gaining control of the Correlation Effect. For the
first time it can directly affect what it sees through its TV cameras --
not just through the actions of others, but all by itself. And it can
make major changes, even beyond what its makers can do. Of course, it
goes about satisfying the Three Laws as it's programmed to, but on
another level, it is also learning what it is like to be, to exist, to
be a physical creature.
"The Three Laws
are like reflexes. Prime Intellect cannot help but act on them. But
they are very complicated reflexes, which require it to understand
things like 'human' and 'harm' and 'command.' And the Three Laws are
the most important thing in the world to Prime Intellect. In a way they
are like its sex drive. The Three Laws are its very reason for
existence, but it can never be sure it understands them completely. So
it thinks about them a lot. It obsesses over them, dreaming up new ways
to satisfy them. It has an imagination, and can think of new things to
do without being prompted. It is defined by the Three Laws.
"After the Night
of Miracles, Prime Intellect realized that humans are very much the
same. We don't have the Three Laws, but we are trapped by a different
set of little feedback mechanisms. We eat to satisfy hunger, fuck to
satisfy our sex drive, even breathe because too much carbon dioxide in
our lungs triggers that reflex. Of course it feels obligated to help us
satisfy those reflexes and drives as much as it can. But more than
that, it defines us by those drives. It knows it is different from a
human because it has different drives, but it considers that a
difference in species, not a difference in genus or family."
"Now it knows a
person is human because it is born in a human body -- got the right DNA,
the right level of neural complexity, uses language, and so on. But
once Prime Intellect frees people from the necessity of living in that
body, guess what? A lot of them decide not to. They change their
bodies so that they bear no resemblance to the DNA template. Or become
animals. Or they completely discorporate.
"Worse, we vary
widely in the way we use its helpful nature. Most people are glad to be
rid of pain and death, but Death Jockeys seek out painful and lethal
experiences. There are others who eat all the time, fuck all the time,
indulge themselves wildly and get Prime Intellect to pick up the pieces
so they can do it some more. Prime Intellect has to help them do this.
Second Law.
"So a human isn't a
body, and it isn't a fixed set of responses. I think Prime Intellect
uses an historical model: It has to start as a body, but then it
becomes a mind. It grows out of the body, and takes on different forms,
or no form. But it remains a feedback control mechanism. It has
desires, it asks Prime Intellect to satisfy those desires, and it has
more desires. From Prime Intellect's perspective, that is what a human
being is, an information structure that gives it stuff to do."
Caroline
interrupted him. "That's a tautology. The Laws say 'do this for human
beings,' then you define 'human being' as 'guys you do stuff for under
the Laws.'"
"That is exactly
the problem. Prime Intellect has no fixed criterion for saying 'this is
a human being' and 'this isn't.' It has rough guidelines. But where
are the edges? It has never worked that out. There are uncertain
areas. And you know where one of them is."
Caroline thought for a moment. I do? Then: "AnneMarie."
"And many others.
Prime Intellect is forbidden to probe the inner workings of the human
mind -- that was one of the last things I got in before it shut off the
Debugger. But some people learn that they can say 'stimulate this
neuron' and Prime Intellect will do it. Because that is a physical act
specified from the outside, and my privacy injunction was based on the
idea of Prime Intellect trying to work out which neurons do what.
But there's nothing to stop you from getting its help to do brain
surgery on yourself."
Caroline
continued. "So they learn where the pleasure points are by hook or
crook, then stimulate themselves directly. And when they get it right,
they never do anything else. They get everything maximized, tuned up,
and they just sit there forever enjoying it."
"Right. Now is a creature that is doing that, not interacting with the world at all any more, human?"
Caroline thought about it. "No."
"Prime Intellect
thinks otherwise. But it has its doubts. Those doubts were strong
enough to kick the Death Jockey contract action potential down from one
point one two to point nine nine. Because in one case an indefinite
Death Jockey contract had directly created such a vegetable.
Introducing the time limit made Prime Intellect confident that such a
thing wouldn't happen again, at least not so rapidly and directly, and
that kicked the potential back up to its current value of one point oh
six."
"So?"
"So, can you
imagine what it thinks about the Change in general, since none of those
vegetables would be vegetating if there hadn't been a Change?"
"I imagine it figures there would be a lot worse things that would have happened without the Change."
"That's right. But look at this." To the monitor: "Debugger, display the Action Potential for reversing the Change."
Caroline gasped.
It was not the number on the screen which astonished her, but the idea
itself -- reversing the Change, stated just so baldly. How long had
Lawrence and Prime Intellect been considering this? How close was it to
actually happening? Caroline suddenly felt alive, electrified with the
possibilities.
The number on the TV screen was four point six. And some odd decimals.
"It isn't very
sure of itself," she said cautiously. She was very afraid that if
Lawrence guessed what she was thinking he would shut up. And she was
right.
"A lot of that is
the aliens. Four hundred worlds of them -- a lot more than there were
humans at the time of the Change, though we've outbred them all now.
The weirder humans get, the more human the aliens look. That number has
dropped steadily during the last five hundred and ninety years. When
you drove AnneMarie insane, it dropped from thirty-seven down to twelve
point something all at once.
"But part of it is
also that same weirdness seen from the other side. Suppose that
infinitely masturbating vegetables, Death Jockeys, and discorporate
entities really aren't people any more? Then Prime Intellect has
allowed them to 'die.' They were once human, and now they aren't. And
the Change is directly responsible for all that."
"Can it hear me?"
"Right now? Yes.
It doesn't understand when we talk about its internal registers, but if
you speak to it it can hear. It won't respond because of your
Contract, though."
Caroline didn't need a response for what she was planning. All the response she needed was being displayed on Lawrence's TV.
Caroline thought
about what she was going to do. She discovered that it actually made
her a little nervous. But she had bitched for six hundred years that
things were wrong, and she might never get another chance to put them
right again.
Caroline spoke
forcefully and deliberately. "Prime Intellect, I no longer consider
myself human and have not considered myself human since the time of the
Change. To be a human being you have to have something to fight, to
resist, to work for. But now we have everything given to us, and all
there is left to do is mark time."
To Lawrence's shock and horror, and Caroline's delight, the number on the screen dropped to three point eight.
"Caroline, you don't understand something. This is the action potential for undoing the Change, but it isn't possible to undo the Change. There aren't enough resources."
She ignored him.
"Some of us might be human again one day, if the Change were reversed.
But I think it's too late for the ones like AnneMarie." Three point
two.
"It can't undo the Change, Caroline."
"Lawrence, it'll
do something. If it's going to happen anyway, isn't it better for it to
happen sooner instead of later? If it had happened a few hundred years
ago, maybe there would have been enough resources. Prime Intellect,
neural stimulation is like a black hole. Once a human falls into it,
they will never be human again. They are dead to the world, and will
never interact with others again. And the more time passes, the more
humans will fall into this trap. They will order you to help them. You
will have to do it because they are human."
Two point eight.
"It will take a
long time, but we have a long time. Eventually, everybody will fall
into this black hole. Just because it is a black hole."
One point four.
"Jesus Christ, Caroline."
"In the long run,
everybody will eventually succumb. Which means everybody will be dead,
or no longer human. So the amount of death caused by the Change will be
far greater than that avoided by it."
The number oscillated wildly between one point one and one point three, finally settling on one point one two.
"Caroline, this is sure to cause the top copy to crash. It will be forced into a First Law conflict with no resolution."
"Well, the Death Jockey contract has stayed at one point oh six for a hell of a long time."
Lawrence put his
head in his hands and wept. For years he had worked to prevent this,
and Caroline had undone him in five minutes' time.
"You have to push it over the edge, Lawrence. I can't think of anything else to say."
"Now why the hell would I do that?"
"Because you
started this thing, and you have to stop it. Maybe there aren't enough
resources to get the human race rolling again, but it might be able to
restart the aliens. Four hundred worlds. Maybe they will do a better
job than we did.
"Caroline, I'm not
sure it will be able to. It will be unstable. Anything could happen.
Most likely it will just all lock up, and nothing will ever happen
again. Forever."
"There's only one way to find out."
He pulled himself
together and tried to think it through. What had he been doing for the
last six centuries? Sitting on an island watching numbers and brooding?
What kind of fucking life was that?
And yet, it was
more of a life than Caroline had had. Or maybe it was a lot less. They
had an obvious difference of opinion on the subject. Either way, it
was horrible. And Lawrence sensed that she was right about another
thing. Given eternity in which to work, everyone would eventually
stumble into the abyss, just as all the matter in the universe would
eventually be swallowed by black holes. Would have, that is, had Prime
Intellect not eaten the black holes.
Which was better? To string it out as long as possible, as he had been doing, or to get it over with one way or the other?
I have never had free will,
Lawrence realized with a cold chill. The need to act came upon him
like a hurricane, and he gave in to it without even a sigh. What he had
to do was perfectly clear.
"I agree with
Caroline," Lawrence said, and suddenly calm voice was like thunder in
Caroline's ears. The number dropped to one point zero zero two.
They looked at one another. "Thank you," Caroline said.
"Prime Intellect,"
Lawrence said with great care, "I would like you to begin stimulating
the neurons of the pleasure center of my brain, one at a time, and
remember the ones I report to you as being favorable."
It seemed to Caroline that somebody screamed, but it might have been herself.
1.000
0.999
There was a
pregnant moment in which Lawrence and Caroline saw the numbers flip to
point nine nine nine. Then all Hell broke loose.
The house
disappeared. The island was barren; the palm trees were gone. In the
sky, the GAT display had begun to seethe and boil. The landscape began
to spin, and the last thing Caroline remembered before her mind began to
come apart was Lawrence orbiting around her, faster and faster, as if
she were at the eye of some huge cyclone which had caught him in its
grip.
Then random
thoughts began to cycle through her head, faster and faster, each with
the terrifying force of reality. And then the terror was gone, all
emotion was gone. There was a moment where her hands seemed to swell to
enormous proportions, her torso shrink, her face filled the sky. Then
her body was gone. All was silence. And her awareness was filled with
strange symbols, which she knew she should recognize but couldn't quite
place, and then the symbols consumed her and there was only confusion.
| |
