Natalie B. Hogg

Cosmologist

Find the stable and pull out the bolt

Brown piece of paper covers a computer keyboard. Through a ripped hole in the paper, the keys "A" and "I" are visible

On the use of ChatGPT in academia: a measured response

Almost exactly two years ago, I wrote what might charitably be called a think-piece on the use of ChatGPT in academia, or more specifically, my observations on the use of ChatGPT by a handful of my colleagues in the small niche of French cosmology which I inhabited at the time. On reflection, what I was reacting to was less the use of large language models (LLMs) in research, and more how I felt my colleagues were undervaluing or mistrusting their own creative output. I was writing from the rhetorically precarious position of someone who believes themselves to be creative, values their own creative abilities, and had, at that point, never used or interacted with an LLM in any way, shape or form.

I never claimed to present an unbiased perspective (as I feel is my right, writing on a website of which the domain is my own name), but with the benefit of two years’ hindsight, what comes across to me most strongly is the emotion behind my words, rather than the presence of any well thought-out argument. With this current post, I aim to summarise my opinions about LLM use in a slightly more measured way, and discuss how and why they have changed since that original post.

Note: Since I used a large number of footnotes and some references, I found it more convenient to use LaTeX to typeset this post. You can find the pdf embedded below, and it is also available to download.

Find_the_stable_and_pull_out_the_boltDownload

Natalie Hogg

, , , , , , , , , , , , , , , , , , , ,

3 responses to “Find the stable and pull out the bolt”

  1. The machines are fine. I'm worried about us.

    […] only works when the fence is well constructed, and his is. Natalie Hogg wrote a disarmingly honest essay about her own conversion from vocal LLM skeptic to daily user, tracing how her firmly held […]

    Like

    Reply
  2. Till Sahm Avatar
    Till Sahm

    I am currently reading a lot of literature, essays and opinion pieces (in no particular order or structured fashion) about the use of AI in different parts of our day-to-day life. I came across your post in Minas Marakmanis post which I saw through Hacker News (Y-Combinator).

    The funny thing is that I can agree with your Before-And-After viewpoint of AI. I did live through the same phase. Still I am more a skeptic of AI and adoption of widespread use. I wanted to comment some of your arguments.

    Before-And-After argument for the climate. I think the After-Argument (as you state yourself) is weak, and I don’t think you can have a climate budget. The problem of living in a western state is, that by existing the society around as uses up resources, even if we ourselves try to avoid it, e.g. through veganism. The climate crisis is not an individual problem (which I think has been made popular by BP and “the climate footprint”).

    You even acknowledge this in the “My use of LLMs” and say that your usage was driven by greed. And you strengthen this transparency in the next section by undermining that we should use LLMs to go from programmers who can do science to scientists.

    In the conclusion you seem to give up your reasons for not using LLMs and say verification is the last key in the puzzle for yourself. Please don’t give up your arguments. Also differentiate between LLMs and LLM-products. Using a product is different from a tool by itself. This differentiation is hard for LLMs, because of the sheer mass of computation (training and inference) that is needed and the unavailability for the individual (this also broadens the gap between privileged and non-privileged humans).

    I agree with the conclusion that LLMs are a new tool in the belt, likewise were books, calculators or computers. But it is still good to ask for better tools. Less polluting ones (killing the planet by using their own gas power plant…) and more available ones. A tool is not a thing that has no history. A good tool can become a dangerous one and the other way around. A

    Please don’t see this as a negative review or comment. I rather want to share my thoughts and be part of the discussion. Contact me by mail if you want to talk more about it.

    Like

    Reply
    1. Natalie Hogg Avatar
      Natalie Hogg

      Dear Till,

      Thank you for your thoughtful comment. To continue the theme of honesty, I suppose I have generally felt smug about my (relatively) hardline vegetarianism and my general avoidance of flying, in the same way that I felt smug about not using LLMs for quite some time. But you are correct that climate change is not solely an individual issue, and this smugness/moral righteousness is not actually helpful in any way. In fact, it probably puts people off changing their habits!

      I would say that I also don’t want to go too far in the other direction — towards a kind of helplessness/nihilism that believes the fate of the world is in the hands of BP and Shell, and I personally am powerless to do anything about it. There are meaningful individual actions — litterpicking a nearby riverbank, or planting native flowers in one’s garden — which make a very big impact on a scale of a few centimetres, or metres. It’s not going to change the world, but it is going to make a difference to a large handful of plants, insects, and birds. It’s a hackneyed but useful sentiment: “You are not obligated to complete the work, but neither are you free to abandon it”.

      I think I did give up with my conclusions somewhat. Partly because I felt I’d written myself into a corner — I have these worries, and yet, and yet, I still continue to use. To understand why is a level of personal and social psychology that I am not equipped to tackle nor ready to share, I think. I felt that the piece must have a conclusion, just to try to remain in line with a basic narrative structure.

      You are completely correct to point out the distinction between LLMs and LLM-products. In my ideal world, university departments and companies would train their own small LLMs for internal use — this would improve security, privacy, access, and environmental impact (by distributing the cost). And in fact, the difficulty of training such a model is actually not as much as you might think. Case in point, my group leader Will Handley is doing precisely this: he has a machine with a few GPUs in his office, and is creating a homegrown version of Claude Code for us to use. Time will tell how it compares to the really big models, but for generic programming tasks I expect it will hold up quite well.

      “A tool is not a thing that has no history.” This is a beautiful phrase, and an insightful sentiment. Of course, the complaint that certain tools are too damaging/polluting can be applied to vast swathes of human invention! Steam engines, combustion engines — even burning wood is terrible for the environment! Time and again we have realised this, and come up with solutions. Unfortunately, due to the influence of money on politics, the solutions are usually driven by a notion of making things more efficient rather than less damaging. Sometimes the two coincide. This is where we start to run up against the immovable object of society. We can demand, we can campaign, we can boycott — but without a billion units of a very strong currency, our power as individuals to force change is small.

      So I think this brings me round to my conclusion again; just as I might say, drive your car, but carefully: use LLMs but carefully. And we may see access becoming more and more restricted in the near future anyway, once the venture capital money dries up and the major companies have to start making good on their investments. This would naturally drive a growth in open-source, homegrown LLMs, which I think would be better all round. Or perhaps, to coin a phrase, SLMs — small language models!

      Thank you again for your comment.

      Like

      Reply

Leave a reply to Till Sahm Cancel reply