Dmitry Kudryavtsev

Dead Internet Theory

Sun, 18 Jan 2026 19:00:00 GMT

The other day I was browsing my one-and-only social network — which is not a social network, but I’m tired of arguing with people online about it — HackerNews. It’s like this dark corner of the internet, where anonymous tech-enthusiasts, scientists, entrepreneurs, and internet-trolls, like to lurk. I like HackerNews. It helps me stay up-to-date about recent tech news (like Cloudflare acquiring Astro which makes me happy for the Astro team, but also sad and worried since I really like Astro, and big-tech has a tendency to ruin things); it mostly avoids politics; and it’s not a social network.

And, in the fashion of HackerNews, I stumbled upon someone sharing their open-source project. It’s great to see people work on their projects and decide to show them to the world. I think people underestimate the fear of actually shipping stuff, which involves sharing it with the world.

Upon glancing at the comment section, I started to see other anonymous participants questioning the validity of said open-source project in terms of how much of it was AI-generated. I grabbed my popcorn, and started to follow this thread. More accusations started to appear: the commit timeline does not make sense; the code has AI-generated comments; etc. And at the same time, the author tried to reply to every comment claiming that they wrote this 100% without using AI.

NOTE

I don’t mind people using AI to write code, even though I tried to resist it myself, until eventually succumbing to it. But I think it’s fair to disclose the use of AI, especially in open-source software. People on the internet are, mostly, anonymous, and it’s not always possible to verify the claims or expertise of particular individuals.

But as the amount of code is growing, considering that everyone is using AI to generate whatever-app they want, it’s impossible to verify every piece of code we are going to use. So it’s fair to know, I think, if some project is AI generated and to what extent. In the end, LLMs are just probabilistic next-token generators. And while they are getting extremely good at most simple tasks, they have the potential to wreak havoc with harder problems or edge-cases (especially if there are no experienced engineers, with domain knowledge, to review the generated code).

As I was following this thread, I started to see a pattern: the comments of the author looked AI generated too:

The use of em-dashes, which on most keyboard require a special key-combination that most people don’t know, and while in markdown two dashes will render as em-dash, this is not true of HackerNews (hence, you often see -- in HackerNews comments, where the author is probably used to Markdown renderer turning it into em-dash)
The notorious “you are absolutely right”, which no living human ever used before, at least not that I know of
The other notorious “let me know if you want to [do that thing] or [explore this other thing]” at the end of the sentence

I was sitting there, refreshing the page, seeing the author being confronted with use of AI in both their code and their comments, while the author claiming to have not used AI at all. Honestly, I was thinking I was going insane. Am I wrong to suspect them? What if people DO USE em-dashes in real life? What if English is not their native language and in their native language it’s fine to use phrases like “you are absolutely right”? Is this even a real person? Are the people who are commenting real?

And then it hit me. We have reached the Dead Internet. The Dead Internet Theory claims that since around 2016 (a whooping 10 years already), the internet is mainly dead, i.e. most interactions are between bots, and most content is machine generated to either sell you stuff, or game the SEO game (in order to sell you stuff).

I’m ~~ashamed~~ proud to say that I spent a good portion of my teenage years on the internet, chatting and learning from real people who knew more than me. Back in the early 2000s, there were barely bots on the internet. The average non-tech human didn’t know anything about phpBB forums, and the weird people with pseudonyms who hanged-out in there. I spent countless hours inside IRC channels, and on phpBB forums, learning things like network programming, OS-development, game-development, and of course web-development (which became my profession for almost two decades now). I’m basically a graduate of the Internet University. Back then, nobody had doubts that they were talking to a human-being. Sure, you could think that you spoke to a hot girl, who in reality was a fat guy, but hey, at least they were real!

But today, I no longer know what is real. I saw a picture on LinkedIn, from a real tech company, posting about their “office vibes” and their happy employees. And then I went to the comment section, and sure enough this picture is AI generated (mangled text that does not make sense, weird hand artifacts). It was posted by an employee of the company, it showed other employees of said company, and it was altered with AI to showcase a different reality. Hell, maybe the people on the picture do not even exist!

And these are mild examples. I don’t use social networks (and no, HackerNews is not a social network), but I hear horror stories about AI generated content on Facebook, Xitter, TikTok, ranging from photos of giants that built the pyramids in Egypt, all the way to short videos of pretty girls saying that the EU is bad for Poland.

I honestly got sad that day. Hopeless, if I could say. AI is easily available to the masses, which allow them to generate shitload of AI-slop. People no longer need to write comments or code, they can just feed this to AI agents who will generate the next “you are absolutely right” masterpiece.

I like technology. I like software engineering, and the concept of the internet where people could share knowledge and create communities. Were there malicious actors back then on the internet? For sure. But what I am seeing today, makes me question whether the future we are headed to is a future where technology is useful anymore. Or, rather, it’s a future where bots talk with bots, and human knowledge just gets recycled and repackaged into “10 step to fix your [daily problem] you are having” for the sake of selling you more stuff.

Thoughts on AI, Intelligence, and Knowledge

Fri, 29 Aug 2025 19:00:00 GMT

I have written extensively about AI in the past: Vibe management, The commoditization of AI, Living side by side with an AI, ChatGPT, AI, and the future of tech, and recently In the name of progress. And most of these articles are somewhat negative towards AI.

Now, I’m not gonna lie and tell you that I never use AI, and all of it is bad. I do use it. Instead of Google search; in order to do product comparisons; plan trips and brainstorm ideas; check for facts (that sometimes I cross-reference with other sources). But I never used AI for coding. Sure, I used AI to ask for help like “how do I sort an array in Rust”, or “how to split a string in Ruby”, and sometimes even requested pieces of code. You know, the usual stuff, the same stuff one would Google for before AI, but now will land you on StackOverflow question with 52 people arguing what is the best over-engineered way to sort an array in JavaScript.

But I never let AI write code for me. I was in the camp of “if you don’t do it yourself, you will lose this ability”. And recently, my friend told me that all is lost, because I confessed to him that I now started to let AI write code for me, while I take the wheel of reviewing the code, and steering the AI towards the accepted conventions, and optimal solutions. And… I like it. But I still have this internal debate between the angel and the devil who each whispers into my ears two different ideas. And as always, when I feel lost or confused, I write about it, because writing is magical.

What does it mean to do something from scratch?

One common argument that I see online, and have expressed myself, against AI — is that you no longer create things from scratch. If you let AI write the code, are you really a programmer then? This question has been asked throughout history in all different variations. It was used as an argument against the use of high level languages — are you really a programmer if you don’t know how binary sort works, but instead you use whataver_array.sort()? Are you really a programmer if you copy string by assignment, a = b, instead of manually allocating the needed memory, and terminating it with \0? Do you really know what does it mean to program a computer to do something if you never xord a CPU register?

I saw a video posted on HackerNews recently, where a guy created a floppy disk from scratch. One of the comments went along the lines of “cool project, but is it really from scratch?”, implying that the author did not obtain the needed chemicals himself, mined the necessary metal himself, or created the plastic material needed to print the housing. This argument can be applied in many different aspects in life: do you really build your own house if you did not bake your own bricks, and cut your own lumber? Does a home-made dress, really home-made unless you planted and harvested all the cotton, and produced the sewing tools yourself? Where does one draw the line of what does it mean to do something from scratch?

What does it mean to be intelligent?

Another common argument in favor, or against, of AI, is the argument of intelligence. When I do something and I fail, I resort to reasoning and understanding my actions with an attempt to improve my next iterations. Contrary to what most people believe, we learn not by reading books, but by doing things, failing, and iterating. Reading books without doing is acquiring knowledge. Applying said knowledge is the definition of intelligence. And one’s intelligence is built on the intelligence of others. A cook who makes a tasty dish does not necessarily know how to grow or harvest the needed ingredients for said dish, yet he is still intelligent. Therefore, using AI in order to produce a working code, does not disqualify me from being an intelligent being.

Or maybe it does? Where do we draw the line? Going back to the chef example, while he does not grow and harvest the ingredients, he most likely has knowledge of their tastes and, through trial and error, has found a combination of ingredients that produce a good tasting dish. He could have asked ChatGPT to create a “tasty dish”. The result would, most likely, be bad. And hence, he would iterate: “I want it more sour”. And so on. Does this constitute as iteration in the sense of an intelligent being? Thomas Edison has iterated for a thousand times before creating the light bulb. With each iteration he came closer to a working product. What if instead he would prompt LLM to create a light bulb with a thousand iterations, would it qualify him to be an intelligent inventor of the light bulb?

Moreover, people tend to over-romanticize the meaning of the word “creation”. If you go to a bunch of restaurants, you will, eventually, notice that most dishes are the same. Sure, some restaurants might bring their own touch (sometimes bad), but in general, a hamburger is more or less the same in most places. Similar thing can be said about cars for examples. Most cars look the same. Cars like VW Transporter, Ford Transit, Mercedes-Benz Vito, and Peugeot Expert — look more or less the same. When people claim to be creating a software, they tend to overlook the fact that a good chunk of the code in there was not written by them, but rather introduced as third-party dependencies, or algorithms from Wikipedia / blogs / StackOverflow. And then we go deeper and deeper the rabbit hole, until eventually we ask “did you really create something if you had to rely on the works of hundreds of thousands of people throughout thousands of years of progress”?

Does this mean that all these examples lack intelligence? Just because you take a successful burger recipe, or a car platform / chassis and replicate it, does this mean you did not create anything new?

The lost knowledge

There is a very interesting talk from 6 years ago titled Preventing the Collapse of Civilization, in which the author brings examples of technologies from Byzantine Empire and the Bronze Age — which are now lost. Lost, because nobody documented how they were built, and the people who built them did not pass that knowledge down. The talk then goes into discussing how something similar might happen in tech. A common example people like to bring is the COBOL language. COBOL is still used heavily in the banking system, yet the number of COBOL developers is declining. What will happen when there will be no more COBOL developers to maintain existing systems written in COBOL? This will become lost knowledge.

The same argument is applied for AI. As we move higher and higher in abstraction levels, we lose the ability to understand the lower levels, and eventually, knowledge will fade from existence as people will have no incentive to acquire it. Why would one learn the various sorting algorithms, and how they work, if LLMs will be capable to offer the best solution for a particular use case? How many developers truly know what sorting algorithm is used when they call .sort() in their preferred programming language?

Too far-fetched for you? There is a confirmed decline in people, especially among Gen-Z, who can write by hand (Oxford Learning, Wired). As we move more towards digital life, the need to write by hand declines. It is possible that in the near future handwriting will become lost knowledge.

But does it even matter?

When I was in my mid-twenties, my friend and I used to discuss the following question: If I take you and teleport you back a thousand, or two thousand, years, what would you be able to teach the people of that time from our modern knowledge? It’s an interesting though experiment in order to understand what knowledge is. You could say: Ha, its easy, I just print the entire Wikipedia and go back in time, and then I can teach them everything we know. But how would you talk to them? I bring a dictionary. But is this your knowledge then, or are you just a vessel? With the same luck, I can teleport a dictionary and a printed version of Wikipedia, and let the ancient people figure it out themselves.

Any knowledge we possess always stands on knowledge of someone else. I know how to write a program in Rust or JavaScript, yet I don’t know how to build a CPU that will execute this program. A person who knows how to build a CPU, might not know how to obtain the needed materials for it, and so on.

My negative view towards LLMs was motivated by my moral superiority of “well, at least I know how to actually write a software”, which is in part true, but in part a self-deception. I just defined what does it mean to know how to write a software, while taking for granted everything else outside my definition such as how build a CPU, how it works, etc. Just like the guy with the floppy disk who defined “from scratch” as taking material and a 3D printer, but obviously there were people for whom “from scratch” means taking (or building?) a pickaxe, and go mine your own minerals.

But in the end, does it really matter? There is a theory I’ve heard about that everything in life in cyclic. The reason we engage in wars, despite the fact that we have documented horrors of wars; the reason we fall to the same old scams which are adapted to modern times; and the reason that old styles and technologies are coming back — is because everything is cyclic. We are not really capable to learn from past experiences, but instead each generation has to take the same path, and make the same mistakes. We oscillate from liberalism to oppression, from freedom to lack thereof — because despite the fact that we have knowledge of how bad (or good) things have been, we do not have the experience of it. And then there are people who are caught in-between transitions. People who have experienced the bad, the good, and are on a path to experience the bad once again, not understanding how it is even possible that those damn young people ignore it, because back in my days…

In the end, it doesn’t really matter. Every opinion we have for or against any technology, is just that, an opinion. People take a stance according to their moral views and beliefs about the world. Such as myself. I guess I could say that I believe in meritocracy, despite the fact that I know that society is not built on merit. But it does not prevent me from believing that a good software engineer is someone who writes code by hand and understands the meaning behind every statement. And it’s just my world view, with boundaries that I’ve created to justify it. Boundaries that say that it’s not important to understand how the hardware works, or the chemistry behind it.

The only constant thing in life is change. Everything is bound to change. Even documented history can be shaped to whatever narrative you want. Knowledge, especially in digital age, rarely gets lost. Handwriting techniques are still there, every Gen-Z can go and learn them, they just don’t need it. The same way we no longer use Papirus to write, or wax seals to signify the authenticity of a document, because we found easier ways to do it.

I guess what I am trying to say is that gate-keeping a certain aspect of our society has no benefits. The reason Gen-Z don’t write by hand is that they have no need for it. It’s the aristocracy all over. People who have comfortable life tend to sit and discuss morals and ethics, while the common-folk try to survive day by day. Difference being that in the old time, aristocracy has been, mainly, passed from generation to generation. Children of wealthy people learned to read and write because their parents knew how to read and write, thus becoming aristocrats. Nowadays, knowledge is mostly available to everyone. Sure, using AI might dumb-down your critical thinking skills, but nothing prevents you from writing code by hand in your spare time in order to keep your mind sharp. I still read books, despite the fact that the general trend seems to indicate a decline in book reading.

And I can go on and on about such examples. We lost the art of fixing our electronics or cars, for the sake of outsourcing the labor to someone else, be it a car mechanic or a factory in China, and this made us less capable. We lost the art of growing and harvesting our food, because we have the farmers and their machinery to take care of that, and this made us less capable. We have surrounded ourselves with warmth and comfort, thus prolonging our lives, and this made us less capable and resistant to the harsh environment.

And I guess the topic of “X is making use dumber / replaces thinking / etc” has been discussed from the first day we gained the ability to philosophize and think. The only way to prevent X from making use dumber, is to go back to hunter-gatherer tribes whose sole goal in life is surviving yet another day, and reproduction.

Vibe Management

Tue, 01 Apr 2025 16:00:00 GMT

Vibe coding is taking the internet by storm. Popularized by Andrej Karpathy, it changes the entire industry. But why only engineers should have all the fun?

Andrej Karpathy Tweet

I’m in favor of making as many professions in tech redundant, as possible. I mean, if I can enjoy singularity while vibing my way through my code, why upper and middle management shouldn’t? And the reality is they should! And they can!

As of today, both ChatGPT and Claude are more than capable of replacing your average middle manager such as team lead or product manager. Here is an example of an interaction between a product manager and a potential customer that begs for a feature:

You are a product manager, and a customer requests a certain feature to be implemented. The truth is, you are over capacity, and don't really want to implement it, but it is your duty to make your customers happy, it's in the DNA of your company "Customers First". Write a short statement that will make the customer feel good, without making any commitment about deadlines, or even if the feature will be delivered at all.

Thank you for sharing your feedback! We truly appreciate hearing from our customers, as your insights help shape the future of our product. This is an interesting idea, and we’ll be sure to take it into consideration as we evaluate potential improvements. While I can’t provide a specific timeline, please know that our team is always looking for ways to enhance the experience. Keep the great ideas coming—we love hearing from you!

Delightful!

But we won’t stop there. Team lead is our next goal! Coming up with a good way to share negative feedback, is a very challenging thing. It can even harm your mental health, or turn your subordinates against you. But don’t worry, we can remove YOU from the equation, and instead let ChatGPT do the heavy lifting:

You are a team lead of in small engineering team. One of your peers underperforms. It is your duty to deliver negative feedback to them, without hurting their feelings or demoralizing them even more, as well as protecting yourself from getting labeled as a bad manager. Write a short statement that you can say during 1-on-1 with your subordinate, to make sure they will understand that they need to start performing, or there is a big line of candidates that can replace them.

Hey [Name], I wanted to take some time to check in with you. I’ve noticed that some of your recent work hasn’t quite met the expectations we set, and I want to understand if there’s anything blocking you or if you need support in any way. My goal is to see you succeed here, and I know you’re capable of more. That said, we do have high standards, and I need to see improvement moving forward. Let’s work together to make sure you’re on track because this is a competitive environment, and I want you to be in the best position possible. What can we do to help you step up?

Now that’s a banger! Who wouldn’t want to dedicate 24 hours a day to their corporate overlord after hearing such constructive and thoughtful feedback!

Let’s take it a step further. When there is no middle management, we can argue for the necessity of upper management. Undoubtedly, but CEOs perform a very important role in the company. But the truth is, with the help of ChatGPT, the CEO can forget that problems exist, and embrace exponentials. They no longer need PR teams, and they can create infinite number of iterations before they land on something they like. And they don’t even have to touch the keyboard, they can talk to Whisper. They can run a company, but it’s not really running a company — they just type stuff, copy stuff, and see stuff, and it mostly works:

You are the CEO of security company called CrawlStroke, and earlier today, your company have shipped a bricked update that broke the entire internet. Blue screen of death is present at airports, hospitals, emergency service, banks, etc. Everyone is angry, and shareholder are not happy. Write an apologetic press release, with emphasis on taking full accountability, fixing it as fast as possible, and offering a 10$ discount for Uber eats to your customers.

SUNNYVALE, Calif. — [DATE] — This morning, CrawlStroke issued a routine security update that, due to an unforeseen technical error, resulted in widespread system failures across multiple industries. We recognize the severity of this incident, and we take full accountability for the disruption this has caused to businesses, critical infrastructure, and individuals worldwide.

First and foremost, we deeply regret the impact this has had on hospitals, emergency services, airports, financial institutions, and countless other organizations that depend on stable, secure systems. Keeping the world safe from cyber threats is our mission, and today, we failed to uphold that responsibility. For that, we are truly sorry.

Our top engineers have been working around the clock to resolve the issue. We have already deployed a fix to affected systems, and we are providing direct assistance to organizations still experiencing issues. If your business is still impacted, we urge you to contact our emergency response team immediately at [support hotline/email].

While we cannot undo the disruption this has caused, we want to make things right. As a small token of our apology, we are offering a $10 Uber Eats credit to our affected customers. We know this doesn’t fix everything, but we hope it brings some relief during this challenging time. Details on how to redeem this offer will be provided via email.

We are conducting a full internal review to understand how this happened and to ensure it never happens again. Transparency and accountability are core to our values, and we will share our findings once our investigation is complete.

We appreciate the patience and resilience of our customers, partners, and the global community as we work swiftly to restore normalcy. We will provide ongoing updates at [company website/social media].

Sincerely, [Your Name] Chief Executive Officer CrawlStroke

OMG 🤩! Who wouldn’t want to make business with such an accountable CEO?

So here you have it! It’s a new kind of management and leadership that I call “Vibe Management” and “Vibe Leadership” where you just give in to the vibes. And I predict that it will replace 90% of upper and middle management in the next year, and within five years, all upper and middle management will be able to join us, software engineers, in the singularity, while all of us will be just vibing our way through our careers!

Bright future ahead of us!

The commoditization of AI

Thu, 30 Jan 2025 18:30:00 GMT

Earlier this week, a Chinese company released a new AI model called DeepSeek, that shattered the big tech giants that tried to own AI. Seeing this, brings some memories from the past.

In the 1990s, there was a tech revolution called “The Internet”. According to Our world in data, in 1990 there were about 2.6 million of global internet users. By the end of the decade, this number increased by more than 15,000% up to 416.2 million of global internet users. From that moment on, the increase was almost exponential. One of the companies that caught this trend of “the internet” was Cisco, and around 1997 their stock started to climbed and in March 2000 it’s price reached $82 per stock, putting it at a total market cap of $536B. This was the highest stock price, and market cap, that Cisco has ever reached. It stayed there for a year, until crashing back to $15 per stock on March 2001. Today, it’s stock trades at around $60, and its market cap is about $230B. What caused this fast climb and sharp decline, you might ask?

The 2000s were an interesting time, when the entire tech world has lost its mind, which eventually led to the dot-com bubble bursting. Seeing this new shiny thing called “the internet” becoming accessible to more and more people, led investors to seek the next unicorn to put their money in. Cisco was an established networking equipment company, and it continued to expand its offerings by acquiring other companies that were doing networking, voice over IP (VoIP), digital video, LAN switching, etc. And despite how much tech bros and investors will sell you the “we want a better world for everyone”, what they truly want (at least as number #1 priority) — are profits for themselves. Hence, they were looking for someone to “own” the internet. What they failed to account for is that the internet will become a commodity, which it did. But what it has to do with DeepSeek and AI?

You see, the entire AI tech bubble is built on the assumption that doing AI is hard, and it requires billions of dollars of investments. Just few days ago companies like Microsoft, OpenAI, and Oracle announced project Stargate which will require a $500B investment in order to build nuclear power plants, data centers, and acquire hardware to train AI models. AI seemed to be out of reach for the common folks, forever to be in the hand of the big corporations with deep pockets, who will profit massively from it. But then came DeepSeek, which released a model that is at least as capable as the top model from OpenAI, if not more capable. They not only set a new bar for AI models, but they also released the model as open source, and claim to have trained it on older hardware for less than $6M compared to the billions that US tech leaders are pouring into AI.

This caused Nvidia to lose almost $600B of market value, a 17% drop, one of the biggest market loss in history. There is no simple, or logical explanation as to why this happened, because Nvidia still seems to be the major provider of GPUs for AI purposes, and it’s not like AI is going anywhere. Nvidia had a sharp climb since mid 2023, just 6 months after releasing ChatGPT, so this might be the market correcting itself. Or as I always said, nobody knows what the stock market is, and why things happen as they do. And while DeepSeek claim to have used older hardware, which could be a plausible claim, there is no real way to validate their claim as China not supposed to have access to modern GPU chips due to export bans, and they might have inclination to hide such information.

Back in the early 1990s, the internet was not accessible to most people. Some businesses invested into getting equipment in order to be able to access the internet, and a select few from the public used to visit Internet Cafés — public places with computers connected to the internet. Personal computers were often sold without the expensive addition of Ethernet cards that allowed your computer to be connected to the modem. Moreover, adjusted to inflation, the cost of getting a modem varied anywhere between $600 to $1000, and that’s in addition to having an internet capable computer. In short, this technology was out of reach for most people. Fast-forward to today, Ethernet is built into every single computer, and you can get a router for as little as $50.

Just like with the internet, AI is going to transform. From being tightly controlled by a group of tech people to be run on expensive hardware, to being freely available and run on consumer grade hardware. There are people today who can run the most capable DeepSeek model at home. Of course, it is not affordable to many yet. Most use a cluster of 6 to 8 Mac Minis which can easily reach the cost of $10K; and those who run it on PCs, require server grade motherboards that support hundreds of GBs of RAM, as well as Nvidia GPUs which are price hiked at the moment.

But make no mistake, we will eventually arrive at a future where AI will be run locally on your devices. With Apple investing into Neural Engines in their M-series CPUs, to dedicated AI boxes from Nvidia and AMD, we will get there. And just like with the internet, the AI bubble will burst, and AI will become a commodity. You probably won’t be able to train an AI model at home, as it will still require money and hardware, but just like with the internet, you are not required to lay down your own undersea cables in order to speak with your friends on the other side of the world.

But what about AGI?

AGI, or Artificial General Intelligence, is something that many of these AI companies claim to move towards in their development. Thus, the first one to reach AGI, will be the first one to set the rules. It’s like the atomic bomb race — you either get it first and set the rules, or someone else does. AGI, if achieved, will be a big change in the way the world works. And so many of these tech companies seem to incorporate into their value the promise of reaching AGI. A promise that we’ve heard to come alive next year… every year. And yet there are no signs for AGI, and I don’t think we will see it, at least not as a by product of LLMs.

When ChatGPT came out I, like many others, have seen it as this magical thing. I didn’t really know how to treat it, or how it works. My view of LLMs swung all the way from “they are bad and stupid” to “they are good and useful”, and with every new model, and every new use-case, my opinion is still shifting. But it really clicked for me when I decided to learn, at least high level, what GPTs are and how do they work. I highly recommend this video by 3Blue1Brown about what are LLMs and how do they work.

And after viewing this video, I now see LLMs for what they are — a mathematical model to predict the next most probable word/token. This not only broaden my knowledge, but it also allowed me to use and prompt LLMs in a more efficient way, as I now understand what are their strengths and limitations. But what it has to do with AGI?

For as long as I remember myself, I was bothered with the hardest question of humanity: what is free choice and is it truly free, rather than preselected/preprogrammed by the environment? In short, can we predict certain human actions/interactions by observing/learning about their environment. I don’t have the answer, but it is an interesting thought experiment nevertheless, and it aligns perfectly with LLMs and AGI. You see, LLMs are predictable. I tried both ChatGPT and DeepSeek for the past couple of days, and both outputted similar answers to some of the question(s). Sure, some models perform better in specific tasks like coding, but the general gist is that they can’t think in a sense a human can. Nor they step outside their mathematical model.

Humans are predictable as well, but the difference between human behavior and a mathematical model is that we can break the rules. The other day I was sparring with my boxing coach. He threw a jab followed by a right hook, which I blocked. Then, he repeated the combination, which I blocked. He did it again, which I blocked again. And then he threw a jab, followed by right hook (which I blocked), followed by left hook which shoot me to the ropes. In a sense, he conditioned me by his behavior (input data) to a certain reaction (output data), which can be seen as a training of an LLM. For the same input, he got the same output, albeit a bit different as I was anticipating the combination, thus performing better/different defense sequences. However, him being an experienced fighter, he was able to see and read my reaction in order to take advantage of my situation. Should I have reacted differently, or been a more experienced fighter, he would probably fail to throw me to the ropes.

But boxing is not a mathematical function, and it requires many skills in coordination, reading the environment, and fast reaction. And there is always an element of surprise, or irrationality. This can be seen as randomness by the outside observer, but it’s not. It’s something in a though process of the brain where people are able to break outside the box (which LLMs are constrained to). Moreover, it’s something more than thinking. I don’t like to talk in meta terms (meta the word, not meta the company), but intelligence is not only the processes that happen inside our brain, but it’s also the effects of environment. Things like reading and understanding a situation, gut feeling, etc. Think about a time when you felt embarrassed to do or say something, or when you reacted instinctively in order to avoid danger. I’ve seen it many times throughout history: creative approaches to job searching, advantages on the battlefield, how certain political figures act, etc. And I don’t think that the current state of LLMs will lead us to AGI.

Don’t get me wrong, GPTs are an amazing piece of technology and research, but they are still a mathematical function, once you understand how they work. There is no magic or thinking behind that. And they are certainly not able to break outside the constraints of their mathematical function.

Living side-by-side with an AI

Wed, 14 Dec 2022 13:00:00 GMT

2022 is, undoubtedly, the year of the AI. It started with DALL-E, then GitHub Copilot, and now—ChatGPT. AI is here to stay, whether you like it or not, and we ought to find a way to live with it.

Earlier this month, the OpenAI team released ChatGPT—a GPT-3 based AI chat assistant that can do a lot of things like drafting a blog post idea, writing essays, explaining various topics, writing code etc. I’ve shared my initial impression about it in my previous blog post—ChatGPT, AI, and the future of tech. In the past week, I’ve used ChatGPT to perform some mundane tasks, which I usually use Google for. In this post, I want to raise some questions on how we, humans, are going to co-exist with AI.

Immediate impact of ChatGPT

In my previous post, I said that I don’t think software engineers or content creators will lose their jobs, at least not in the near future. I still stand behind this opinion. However, as with any technological progress, there are those who are going to be impacted—for good and for bad.

When the assembly line was introduced, and later on—perfected with automation, people who used to do what robots do now—lost their jobs. That’s the price we pay for technological advancement. Mundane, and repetitive jobs—are always the first in–line to be replaced. Things like low quality content production for SEO purposes, or researching keywords for blog posts. All these can be easier done by ChatGPT with minor corrections from a human expert.

ChatGPT generates me some keywords for a blog post#859x728

Yet, there are always people who look for low-hanging fruits. In today’s world—content is highly rewarded. No matter what niche you take, be it software engineering or self development, the successful people in that niche produce content. Blog posts, podcasts, YouTube videos, books, courses etc. You can say we do it for ourselves, but the truth is—we all, including myself, want to share our content with the world. Those who write for themselves—keep a journal, not a blog. We all do it for different reasons. Some to learn, others to teach. Some to share with others, others to enrich themselves. I’m not the one to judge, and I believe people should be paid for their knowledge—if they desire so.

However, the thing with new content producers is they think it’s easy. Just slap in some words in a text editor and publish it to WordPress, or film yourself talking to a camera and upload to YouTube. But in reality—content production is a hard job that involves not only the ability to write or speak, but also the ability to distribute your content and build an audience.

And while most of the creators take the hard route of producing content, there are people who will always try to find shortcuts. Like with money making—some people go to work every day, while others try to set up a get-rich-fast scams. And the big beneficiaries of AI technologies today—are those who are looking for shortcuts.

I’m relatively active in communities like HackerNews and Reddit, and I can see the explosion of ChatGPT content on those platforms. I even had an answer to one of my questions, copy–pasted from ChatGPT. Granted, the author of the answer disclosed he/she used ChatGPT. Notion, the highly popular note–taking app, that I no longer use, recently released Notion AI—an ability to generate content right inside the editor itself, leveraging GPT-3 model. They did not work on the countless issues that their users have with the software, such as lack of offline support or E2EE—instead, they’ve put their efforts into AI.

And it’s a smart move because I believe there will be a huge demand for AI, due to the important role of content creation in our current society. Websites, like StackOverflow, have now implemented a temporary policy to ban ChatGPT generated content. Getty bans AI generated content. And I’ve read somewhere that Google is working on, or already have, a policy to punish websites that use primarily AI generated content. I can’t find the source, so don’t quote me on that.

The human touch

From all the examples I’ve seen, and based on my personal experience—ChatGPT is often times wrong. It gets a lot of the things right, but usually make a mistake here and there. And this is one of the dangers of using its raw output. A person, who has no understanding of the topic, can not effectively use ChatGPT. I can ask ChatGPT how a nuclear bomb works, but I’m not a nuclear scientist to verify that the answer is correct. I can, however, go on Quora or Reddit and spam answers at a rate of one per minute, hoping that 90% of them will be correct—while ripping the benefits.

The power of communities like StackOverflow or Reddit—is the positive feedback loop we get from our fellow humans. The ability to audit a solution, or provide multiple answers from multiple people—is priceless in the world of self development and knowledge acquiring. People make mistakes—that’s why we have protocols in place, as well as fellow human beings, to correct our mistakes. ChatGPT is free, for now. If I want to read AI generated content, I can use ChatGPT directly. And so, flooding the internet or the above communities with AI generated content, makes it harder to maintain the quality of the content as well as getting real human answers that come from their personal experience and knowledge.

What is creativity?

When we talk about AI generated content, we need to talk about creativity. If we put aside for a moment, the fact that ChatGPT produces inaccurate content in most cases, one could ask then—what’s the difference between a content written by a human and a content created by a machine?

Creative process is not something that can be described mathematically. Humans, usually, describe their creative process as a-ha! moments. Something like: I was taking a shower, while thinking about the solution to that problem, and then it hit me! I need to use JavaScript!

We then go on and do research on that topic, by reading materials, and eventually summarizing the entire process in a unit of production—be it a system design, a piece of code or art. If we look beyond technical fields, and gravitate towards more creative fields like writing or singing—we can learn that many writers or poets produce content based on their personal experience and background. A complicated child might become a rapper who will rap about his though childhood, which might resonate with children in a similar situation. A WW1 veteran—might write about his horrible experience during the war, as a way to ease his pain and help him cope with that experience, while passing onto the reader the horrors of those events.

The word creativity in itself is associated with something human. AI has no experience. It has no notion of past. It’s not having childhood traumas. Nor it can think like a human can. Therefor, I believe it has no creativity—it can only remaster. And there is something humanly when we look at art or read books and can feel the same things the artist or the author were feeling. I can’t have sympathy towards machine. I usually kick my robot vacuum cleaner when it’s stuck. Please don’t call the robots abuse hot line.

ChatGPT talks about creativity#878x680

Something from nothing

One could, rightfully, ask—Humans rarely create anything from nothing. How is that different from an AI?

I do want to point out that there are great discoveries made by humans. My point is not to say that all we create—is a remix of existing things. But as I’ve said earlier, most of what we do create—is based on past knowledge and experience. A student of Rembrandt will be inclined to create art in a similar style to his teacher—unless he or she decides to explore other styles or artists. The same goes to all other types of content creation. What I write—is based on what I read and the experience I have.

The difference between me and AI—is that I have context and understanding. I know that 2+2=4. A good mathematician, which I am not, also knows why. AI—does not. It simply fed this fact into its neural network. If you feed it with another fact, it will produce answers based on that fact. This creates a negative feedback loop. I’ll talk a bit later about the negative feedback loop.

What is copyright?

When we talk about creativity and remastering—we have to talk about copyright. As I’ve said in the beginning of this post—2022 is the year of the AI. During a very short period of time, something like 3–4 months, we’ve gone from closed-behind-doors-deep-learning-ai-that-beats-humans-in-go-or-chess, towards AI that generates art, code, and poems.

But you know what’s missing? AI that generate music or videos. Yes, there is OpenAI Jukebox, but it didn’t get as much attention as DALL-E, Stable Diffusion, or ChatGPT. While the results of it are amazing—they are not on the same level as other AI tools. And there is no, yet, AI that can generate video content such as movies. I think it’s related to the fact, that existing AI models are trained on existing data. Access to royalty free music or films—is limited. GitHub copilot was trained on public GitHub repositories, while ChatGPT was trained on crawling the internet. Those are huge datasets. GitHub has nearly 128 million of public repositories; while the GPT-3 model was trained on nearly 400 billion websites.

I don’t believe knowledge or art should be safeguarded, unless the author decides so. But I do believe that people have the right to retain copyrights for their work. I’m not a copyright lawyer, so I don’t know exactly where is the line between original work and remixed work. But I know for sure that I can not publish a YouTube video with Britney Spears song in it. This video will be removed, for respect to Britney’s copyright. The same respect that other authors, or open source developers—have. Is it right to train an AI model on an open source project, just because it’s open source? Is it right to train an AI model on a website, just because it’s publicly available? Is it moral to charge money for an app that uses an AI model, which is, most likely, trained on a publicly available art, that generates avatars for you—without giving credit or attribution back to the artists that “inspired” the AI model?

I don’t know. I know there is a lawsuit against copilot. And I know that we must have a discussion as to how we should train those AI models. The fact that some datasets used to train the models—are public, does not mean it can be used without implications, credits, or attributions. The fact that one owns a bunch of legally obtained DVDs with movies, does not grant them the right to open a private movie theater. Images, articles, and blogs posts—sometimes have licenses. And those licenses should be respected by humans as well as AI. And unless we start to raise awareness for this issue, and have an open discussion—we are, potentially, violating the rights of content producers and artists.

Negative Feedback Loop

I’ve mentioned it earlier, and I want to expand on this topic. In one of my previous posts, The software industry is broken, I’ve discussed the fact that software today—is not as optimized as it used to be. One of the reasons I’ve outlined—is the fact that writing software became easier, and in a way dumber. Developers today don’t need to know how memory is managed. There are high level languages that manage it for you. And so, as tools become more available to the masses—they’re ought to be simplified. And simplification, often times, leads to loss or deterioration of knowledge.

A similar thing can, and will, happen to our knowledge if we do not control the spread of AI content. AI is great today because of two reasons—(1) the engineers behind it and (2) the entire knowledge of human beings. Every time an AI generates a piece of code, art, or a text paragraph—it’s because there is work made or knowledge shared by humans, that the AI was able to train on.

I’ve said earlier that, as of today, ChatGPT gets most of the things right, and some things wrong. If we don’t set up protocols in place to distinguish human generated content from AI generated content, AI generated content will flood the internet. Look at how the standards of beauty have changed, because everyone is posting unrealistic photos of themselves, by using a bunch of photo retouching, auto-magical, tools.

There are people who embrace the easy model of producing free content. And by flooding the internet with potentially inaccurate content, we will not only bias the future training process of AI—we will damage the future knowledge of human beings.

The real danger is not that computers will begin to think like men, but that men will begin to think like computers.

— Sydney J. Harris

Conclusion

I tend to think of myself as a realist. Although, if you ask some of the people who know me well—they might say I’m leaning towards pessimism. AI is here. We’ve opened that box. As with every tool ever created by human beings—it can be used for good, and it can be used for bad. People, usually, tend to focus on the good. I’m not going to unwrap my entire philosophical views here, but I tend to, typically, focus on the bad.

We reward content creation. Aside from Google, the 4 most visited websites are YouTube, Facebook, Twitter, and Instagram (source); and the 3 most downloaded apps are TikTok, Instagram, and Facebook. Those are all content creation tools. I’ve read somewhere that today’s kids, when you ask them who they want to be when they grow up, answer a YouTuber or a TikTok/Instagram influencer. And we value knowledge.

ChatGPT, DALL-E, Stable Diffusion, Midjourney, Copilot etc. Those are all great tools for both content creation and knowledge sharing. Every one of us will have to find a place for them in our daily toolbox. ChatGPT crossed 1 million users in less than a week—this is a clear sign that it’s not just a tool that will be forgotten soon.

The striking thing about the reaction to ChatGPT is not just the number of people who are blown away by it, but who they are. These are not people who get excited by every shiny new thing. Clearly something big is happening.

— Paul Graham, Computer Scientist, Entrepreneur, Venture Capitalist, and Author, Twitter

AI here to stay—and more will come. But together with the adoption of those tools into our daily toolboxes—we need to have a discussion about topics such as creativity, attribution, respect for human work and the effects those tools might have on the future of humanity and our knowledge.

#WrittenByHuman

ChatGPT, AI, and the future of tech

Mon, 05 Dec 2022 20:00:00 GMT

Unless you’ve been living under a rock, you probably heard about OpenAI releasing ChatGPT for a free research preview.

After the release, a bunch of different posts across Reddit and Hacker News started to emerge — outlining all the possible reasons why AI and ChatGPT will now replace jobs such as: Product managers, software engineers, tech bloggers etc. Usage of ChatGPT exploded, leading to sites like StackOverflow — to implement temporary policy to ban ChatGPT generated answers (source).

Look — I’m a technology lover. I like messing with technology; I like using technology to improve my life. I automate certain tasks using software; I adore my robotic vacuum and window cleaners, and after getting a dishwasher last year — I don’t understand why I didn’t get one earlier. But, I, personally, can’t share the enthusiasms or doom–like prophecies that we all, are soon, going to be replaced by AI.

Understanding AI

I’m not an AI engineer, and apart from messing a little with artificial neural networks and genetic algorithms many years ago, my AI knowledge is very limited. According to my understanding, AI, and especially the GPT–3 model — built upon feeding on initial content. This means that those models are analyzing existing texts or source code — and learn to generate meaningful content based on that. According to Wikipedia, GPT–3 was trained mostly on crawled websites (source). The famous GitHub CoPilot was trained on source code from GitHub repositories. And so, in my uneducated opinion, current AI models are limited by the knowledge we — humans — have created.

Taking AI for a ride

At first I was skeptical, but ChatGPT was topping Reddit and Hacker News front pages, as well as talked a lot among my colleagues, so I’ve decided to take it for a ride. It’s open for public, for a free research preview: https://chat.openai.com/.

Hey AI – write me a blog post

Before jumping to find whether I’ll be replaced as software engineer, I was wondering will I be replaced as a blogger. On Wednesday, I’m going to publish a new blog post that talks about cursor based pagination. You should totally subscribe to my newsletter to be notified when it’s published, as well as stay updated about any future posts. But in the meantime, I wanted to see — would ChatGPT be able to generate a blog post on that topic. And so I’ve asked AI the following:

write a blog post about cursor based pagination and comparison with traditional pagination, including examples for usage as well as cons and pros of each approach

It generated a nice post, but when it came to comparing pros and cons — the results became messy. It listed 3 pros for cursor based pagination, and while the first two were okay-ish, the third one is a big miss:

It’s more user-friendly. With traditional pagination, it’s difficult for the user to know how many pages there are in total, or to go directly to a specific page. With cursor based pagination, the user can see the cursor value and use it to navigate to a specific position in the dataset.

This is a complete lie. There is an easy way to both (1) know how many pages are there in total, and (2) go directly to a specific page. We’ve been doing it for decades. Moreover, depending on the implementation, the user is not always able to know and use the cursor as he desires. On complex datasets — cursors are opaque and meaningless to the user.

You could say that this is a mistake that can be easily fixed by a human later on, but wait! There’s more! When ChatGPT generated the cons of cursor based pagination, it wrote this:

It’s less predictable. In traditional pagination, the user always knows how many records will be on each page. With cursor based pagination, the number of records on each page can vary depending on how many records have been added or deleted.

The first sentence contradicts the pro outlined earlier. If I know how many records are on each page, I can know how many pages there are. The second sentence — is not entirely true, in fact it is possible (and recommended) to include a limit on the amount of records you retrieve using any type of pagination.

While I am amazed by the ability to generate, generally, good structured and informative post — it’s far from replacing a human, such as myself, who performed a good research combined with personal, and professional experience.

sudo ai – generate me some code

I’m safe with writing blog posts. Let’s see if I can be replaced as a software engineer.

I’ve asked ChatGPT to generate me some code:

Write a function that streams data from external API endpoint and caches it, using typescript and nodejs

And it produced the following result:

import * as request from "request";
import * as fs from "fs";

// Function for streaming data from external API endpoint and caching it.
async function streamData(url: string, cacheFile: string) {
  return new Promise((resolve, reject) => {
    // Check if the cache file exists.
    fs.access(cacheFile, fs.constants.F_OK, (err) => {
      if (err) {
        // Cache file does not exist, so create it and steam data from the API
        const fileStream = fs.createWriteStream(cacheFile);
        request(url)
          .pipe(fileStream)
          .on("finish", () => {
            resolve(true);
          })
          .on("error", (err) => {
            reject(err);
          });
      } else {
        // Cache file exists, so read data from the cache file.
        fs.readFile(cacheFile, "utf8", (err, data) => {
          if (err) {
            reject(err);
          } else {
            resolve(data);
          }
        });
      }
    });
  });
}

ChatGPT suggested using a deprecated module — request. Also, the code itself will fail linting because streamData is defined as async but there is no need for that function to be async, since the code uses callback style fs and has no await statements. Nitpicking, I know.

When I asked ChatGPT, what is wrong with the above code — it was smart enough to output this:

One potential issue with this code is that it uses the request module, which has been deprecated and is no longer recommended for use. Instead of using the request module, the code should use a more modern HTTP client, such as the axios library, to make HTTP requests.
Another potential issue with this code is that it does not handle errors properly. In the case where the request module throws an error, the code does not reject the promise, which means that the error will be silently ignored. To fix this, the code should add a catch block to the request call, and reject the promise with the error if one occurs.

It was able to catch the deprecated request module, but said nothing on the unnecessary async keyword, or suggested to use fsPromises instead of callbacks for a more readable code.

Programming is more than just writing code

For some reason, people keep thinking that a programmer’s job is to write code, while in reality — our job is translating client requirements into code as well as designing, and maintaining complex systems. Writing new, fresh code — is relatively small part compared to maintenance and modification of existing code.

Questions, such as how to sort an array, were long ago solved with all possible algorithms. ChatGPT is no different from Google search for such question, except for being more convenient than performing your own search, while potentially producing wrong or outdated code. The power of communities such as StackOverflow — is that answers are audited and updated. I keep getting notification about new comments or updates to a question I’ve asked more than 7 years ago!

Don’t get me wrong — those are amazing results. I’m happy for the OpenAI team, and the people behind GPT–3 are, undoubtedly, incredible and talented engineers. But I believe that current AI models are limited by our existing knowledge. They are unable to create new things. They are able to remix existing knowledge. Takes bits from different sources, and try to put them in context. Those tools, in their current state, can be a great addition to other tools like searching in Google or using StackOverflow. But I don’t believe we are anywhere close to having an AI that can truly create new things — and thus replacing us in our jobs. Don’t trust me? At least trust an AI, then.

ChatGPT on replacing software engineers#890x417

#WrittenByHuman