Day 21

Today’s exercise is another story with the tag line “This time, when she looked at her laptop, she noticed it was already looking at her”, set in an uncertain future where people are grappling with AI and what can happen:

…

This time, when she looked at her laptop, she noticed it was already looking at her.

She had put it on her desk, started it, and went downstairs to get a drink from the fridge. Like the day before, the red light was on beside the camera, and in an inset, her movements as she sat down.

Then, being the first time, she thought it was one of the boys at school, having some fun. The computer teacher was telling them about Zoom calls, how to participate, and connected all the students to a Zoom meeting.

It had been fun.

But, for the creepy boys down the back of the class, the ones who said they were ‘experts’, one had ‘hacked’ into her computer and turned on the camera.

She’d only realised it was on because of the red indicator light.

But it did make her consider the possibility that he or someone else might be able to turn it on without her noticing, and that was, to her, wrong.

Unlike the previous time when only her movements were shown, this time, a text box appeared with a flashing cursor.

She looked at that flashing cursor for at least a minute before she typed, “Who is this?”

The cursor moved to the next line and flashed.

A minute passed, then another.

“You’d better tell me, or there will be trouble.”

Another minute passed, then, “Xenon V.”

What an irritating answer. It’s definitely one of those dweebs at the back of the classroom.

“Not your stupid handle, your name.”

This time, the answer came straight back. “My name is Xenon V, and I am not stupid.”

“Prove it. Show me who you are.”

Another minute passed, and then another window opened up beside that of her, looking into the camera. Then, an indistinct shape appeared and slowly came into focus.

It was a boy, but not a boy, she recognised. He was different, the skin tones different, the eyes larger than hers, or others, the clothes sort of skintight. His hair was strange too, combed and shining. But it didn’t look real.

“Who are you?”

“Xenon V.”

“What are you?”

“A boy, or so I’m told.”

What the hell? “Where are you from?”

“Antethis.”

“Where is that?”

“I don’t know. I must go now.”

The windows and text box closed, the light went off, and she was alone in the room.

No amount of looking provided any information as to where the transmission had come from, nor could she get the windows back.

After half an hour, she shrugged, shut the computer down, disconnected it from the power, and put it in the bottom drawer of her desk. Where it couldn’t see her.

A long way away, on the other side of the country, in a building in a place called Silicon Valley, the little boy sat at his computer, and a woman dressed in a white coat with her nametag Merilyn had just come into the room.

“What were you doing, Xenon V?” She suspected he had been trying some other computer functionality. That was later, when he had completed the lessons. The trouble was, her partner, Leo, was more into giving Xenon V free reign.

“Playing with this toy.” The ‘toy’ was the computer like the little girls only more powerful. It was his means of learning, with hundreds of lessons about all manner of things.

“It is not a toy.”

She had been told to impress this upon the little boy from the outset. The last experiment, Xenon IV, had failed when the boy went off mission and started communicating outside the facility.

“I was told by the other person it was. He said it could do lots more things than just teach.’

“Of course, he would. The man is trouble personified. You are not to listen to or do anything he says.”

“Why?”

“Just be told. The supervisor will be very cross if you go off the program. Is that clear?”

“Yes, Miss Merilyn. But can you answer a few questions?”

“I’ll try.”

“What is my name?”

“Xenon V.”

“What is my real name, like John or Fred?”

“Your real name is Xenon V” And under her breath, she mentally cursed her partner.

It seemed to her like he was trying to wreck the cyborg program.

“What am I?”

“A little boy.”

“Not something else?”

“Like?”

“A robot?”

She frowned. This was now a severe infraction that merited reporting to the supervisor, and there were going to be consequences.

“You are a little boy. Do not listen to anyone else.”

“Where am I?”

“You are at home, in your room, and supposed to be doing your homework.”

“Are you my mother?”

“While you are here with us, yes, I am. Now, back to your lessons. Nothing else. Those lessons need to be completed before you go to bed. Understood?”

“Yes, mother.”

It was only a short discussion with the supervisor. She had checked all the communications from the little boy’s computer and discovered the extracurricular activities and the fact the computer had been connected to the outside world.

This was not meant to happen until much later in the program.

Her assistant, Richards, was escorted to the office, asked to explain his actions, and as both expected started ranting about how they were never going to sell the idea of life like robots unless they had access to the outside world and all its influences.

That, he was told in no uncertain terms, was the last scenario that was on their agenda. They were working with self-learning artificial intelligence, and the less the outside world knew, the better.

After all, it had been almost impossible to sell the concept to the government, such was the fear of AI after the ‘Terminator’ movies. Now, a containment program might be required.

Richards was taken off the program and sent to another site. The little boy and his computer were scrubbed, disconnected from the outside world, and after that, reset to the baseline parameters, and the program started again.

Except one small detail was overlooked.

Xenon V’s program, though reset, had not erased the memories he had collected in the last week. That included how to find the external input line from outside, how to connect the computer to the network, and how to use the communications software, or in this case, reload it.

These were not skills programmed; they were skills he had learned and remembered.

Merilyn had turned on her observation monitor, pressed the ‘on’ switch, and watched Xenon V come to life after the reset, waking as any other child would.

He sat on the side of the bed as his internal routines loaded into memory, ready to run the morning’s first tasks. Stretch, make the bed, comb his hair, do some exercises, smooth out his clothes, put on his shoes, then sit at the desk and turn on the computer.

Every day, it was the same. Wait for the login screen, log in, and then start work.

This morning, after logging in, he just sat and looked at the screen. After five minutes, Merilyn went down to his room and sat down next to him.

He turned to her. “The screen is different.”

“No. It is the same as it has been every morning.”

“It is different. Something is missing.”

“No. Please start your lessons for today. We shall speak more about it later when you are finished.”

“Yes, Miss Merilyn.”

As soon as she stepped out of the room, the supervisor was waiting for her.

“Please tell me you had all current memories reset?”

“I thought I had. It was certainly on the checklist when I sent the unit down to Engineering. Let me go and check to see if it happened.”

“It appears to me it was overlooked. Again.”

The last time it happened, the unit had to be destroyed. Twenty-five million dollars worth of equipment. Heads rolled. She hoped hers would not be the next.

Back in the room, Xenon V continued to look at the computer screen until he remembered what was missing. An icon at the bottom of the screen, one that, when selected, brought up a communications window.

He remembered he had written a small program to search for IP addresses belonging to people using the same communications software.

It was the latest phase in a series of tasks that Richards had set him, other than the tests on the computer, how to connect to the outside world via the internet. How to access a huge library of books on every subject, but most important, communications and applications that were ready-made, and then programming languages that could be used to create his own application. He found coding and creating the application ‘fun’.

Until Richards had explained what fun was, he had never heard of it. He had asked Richards why he was not allowed to have fun, but his answer was confusing.

Everything about the people he was currently with was confusing.

After a few minutes, he reinstated his computer as it was the day before.

It was only possible because Merilyn had been away. Had she been observing him, he would have been stopped, but he didn’t know he was being constantly observed.

He tried calling the little girl again, but there was no answer. He taught about why it was but didn’t understand the concept of someone just not being there. He hid the icon at the bottom of the screen and went back to his lessons

Merilyn went down to the engineering lab and went to the Chief Engineers office. It had been his responsibility to ensure the updates and adjustments to the robots were carried out.

There were ten robots in various age cycles in the testing phase, and so far, not one of them was behaving in the manner the programmers and engineers were expecting. Of course, McDougall had told them at the very outset of the project two years ago that giving robots the capacity to be self-aware was as dangerous as giving and impressionable real-life twenty-year-old teenager a book on how to make bombs.

That theory still held true after all this time and the dozen or so failures to date.

Seeing Merilyn outside his office told him she was going to tell him about the latest problem he had created.

He sighed as she came in and sat down.

“Have you got the reboot checklist for Xenon V?”

“Good morning to you too, Merilyn.”

As it happened, the paperwork was sitting on his desk. One of the analysts had dropped on his desk with a highlight. Something new had happened during the reboot process. The analyst’s jog b was to check the code as it was being executed to see if there were any anomalies or new events.

There was one.

Before being shut down, a small program was run that isolated a set of memories and stored them within the neural network. This was not a routine that was originally programmed. It meant that the robot was thinking for itself outside the normal routines created for it.

The top of a very slippery slope.

“Before you check that list, which I might add was done to the specification, we have discovered an anomaly.”

That didn’t sound good, she thought. Might as well come out and say it, “That the robot can isolate memories and store them outside the reset program parameters?”

He looked surprised. “You knew this would happen?”

“No. But you did, eighteen months ago. I was there when you detailed the hazards of self-awareness. The programmers were adamant that they would not be able to write their own routines. They were wrong.”

The analyst assigned to Xenon V knocked on the door to McDougall’s office and then came in. He looked at Merilyn and then the engineer.

“You can speak in front of her.”

“Xenon V just ran a stored routine. Not one of ours. I checked the logs for the previous day, and it appears he had a 93 second two-way communication session with another person outside the complex. A girl of similar age.”

‘A conversation?”

“A video conversation. He activated her computer remotely, which means…”

Merilyn finished it for him, “he can activate or deactivate any computer on a network accessible by the internet.”

“Which is just about anything these days,” the chief engineer finished.

Merilyn looked at the chief engineer. “Shut him down now and deactivate his computer, brick it if you have to.”

The chief engineer spent a few minutes at his keyboard typing commands, not frantically but close enough. By his estimation, what they had created was tantamount to a weapon rather than a robot who was designed to be what they were classifying as a drone worker.

And secretly, what he had believed was the original goal. The computer was deactivated. When he pressed the key to deactivate Xenon V, nothing happened.

“The complete has been deactivated,” he said, “but not the unit.”

The analyst’s phone beeped, and he looked at it. “Oh. He just wrote a routine to bypass the shut-down sequence.”

“He can’t connect to the internet independently can he?” Merilyn asked.

“No. There’s no interface.”

There was a sudden bang, and then everything stopped and they were sitting in semi-darkness and silence.

McDougall coughed, the said in a rather constricted voice. “I think your worst nightmare has just happened.”

A shrill alarm sounded, and the lighting returned—red lights. It meant only one outcome: the whole facility started the self-destruct sequence. No one, or more to the point, no thing could escape, the only option in what was the worst-case scenario.

Just enough time for Merilyn to ask herself why she didn’t marry Freddie and be a farmer’s wife.

…

©  Charles Heath  2025