Reflections on “Agentic AI and Research: opportunities and risk”, a SISSA workshop

Cover image by Leo Lau & Digit — https://betterimagesofai.org

On Friday 15th May 2026, I attended the workshop Agentic AI and Research: opportunities and risk, organised by the Scuola Internazionale Superiore di Studi Avanzati (SISSA) in Trieste. I had been kindly invited by Roberto Trotta to contribute to this half-day workshop, and was the first speaker on the agenda. My talk was entitled “Agentic AI for research: friend or foe?”, and I used it as an opportunity to develop my thoughts and opinions on agentic AI previously described on this blog (“Claiming the right to be unhappy” and “Find the stable and pull out the bolt”).

The task of preparing a talk on a topic very different from my own research was a challenging one — not to say laborious, producing a set of slides from scratch covering an hour’s duration. With scientific talks, you have done a piece of research, written it up into a paper, and had it published. Moreover, as a physicist, one is used to talking in terms of hard facts. There is a culturally agreed-upon methodology, and a shared jargon. This makes presenting your own research a relatively comfortable experience; you, and everyone in the audience, is speaking the same language. Of course, people can disagree with your results, or the way you obtained them, but you should yourself be aware of the limitations of your work and be able to defend it.

With the topic of agentic AI, I was on much less firm ground. I have been catapulted into the discourse by virtue of this blog, not a particularly legitimate venue. It’s one thing to write one’s thoughts down and send them into the aether, for the eyes of invisible consumers; quite another to stand in a fluorescently lit room full of real, breathing people and say them out loud. I spent quite a while thinking about my ‘way in’ to the topic; in other words, how to start the talk. What was my hook?

In the end, I decided to start off by talking about cosmology — what it is, and how we do it. As cosmologists, we’re limited to observing a single sky, and to compensate for this, we like to take huge volumes of data and do lots of statistics. Because of this, we’re always looking for ways to speed up our research, to automate — and here we see how the path opens up for the introduction of agentic AI. I liked starting the talk this way — because I was talking to a room comprised largely of non-cosmologists, I could easily find my ‘outreach’ tone; a slow, measured delivery, with plenty of pauses. It also meant that I didn’t have to think too hard about what I was saying for the first ten minutes or so, always a good thing when it’s clear nerves are going to be an issue.

And indeed, nervous I was. For one thing, it was an early start — 9 am, which due to the time difference felt like 8 am to my body — and I am never particularly functional in the mornings no matter the circumstances. I managed to choke down a miniscule bit of breakfast, boarded the bus for the vertiginous journey from the centre of Trieste to SISSA, and arrived in good time. I had been able to check out the room the day before, where some of my fears had been mitigated by the discovery that it was a small-ish, normal classroom, not the spotlighted lecture pit I had been envisaging, which always makes eye contact with the audience difficult. Quite the opposite in fact; the front of the room had a low stage, and even better, a small table instead of a podium or lectern. Complete freedom to move around as I spoke, and a rare height advantage over the entirety of the audience.

I was very pleased with my delivery of the talk. I had a one hour slot, and I spoke for around 50 minutes. I had only run through the entire thing once, and clocked in at an hour and five minutes, but I knew I would say less in the real thing, as indeed I did. Despite the lack of practice compared to what I usually do (in the old days I would run through talks four or five times, though recently that has been decreasing a lot) I felt I said exactly what I wanted to say, in a very coherent manner. I think the only point which didn’t perfectly land was my very final concluding bullet point where I wanted to say something about the increasing importance of scientific honesty and open science in the face of agentic AI; as usual, the issue was trying to over-explain something conceptually very simple.

Happily, there were so many questions that the chair had to stop people asking things so that we could have a break before the next speaker. This was where I felt most keenly the departure that this was from the usual scientific talk. In the usual Q&A session, you can expect to have to explain a certain point in more detail, discuss what you plan to do next, or on-the-hoof try to understand a rambling connection the questioner is trying to make between your work and theirs. Not so here — in this case, I felt more like I was being interviewed, in the sense that the questioners wanted my personal take on things. It’s interesting that I don’t feel this way about scientific questions, in fact; with a science talk, I feel like I could personally be replaced by ‘cosmologist X’, who would have carried out the exact same research as me, and presented it in the exact same way. It feels like I don’t give credence to my own interpretation of my work. Very different to how I felt talking about agentic AI, where everything I say is very much an opinion; measured, I hope, and well-argued, with some grounding in agreed-upon concepts, but an opinion nonetheless.

The most memorable question came from Nikki Arendse. She asked if I felt whether doing research through the medium of agentic AI is more or less fun than doing the research myself. On the spot, this (and many of the other questions) felt difficult to answer honestly, but I had a duty, I think, to the audience and to myself, to do so. And so I said what I felt in my guts — no. Using LLMs feels less fun than doing the work myself. I start to feel like I miss the friction that comes with thinking about a problem, writing notes about it, banging out some code, recording in my work diary the dead ends and blind alleys, beating my head against the endless brick wall that is research. Somehow all of that has narrowed to sitting in front of my computer waiting for Claude’s terminal chime signalling that it’s time for me to press enter again. Maybe I’m doing agentic research wrong, but that doesn’t feel fun to me. I’m starting to wonder if, after much vacillation, I’m returning to my 2024 point of view: that we must shun the use of LLMs. To say nothing of the fact that I’m not actually convinced I’m any faster or better at my job with Claude in hand than I was without.

So much for my talk. After me came Esther Greussing, Research Associate at the Institute of Communication Studies at TU Braunschweig (known historically in English as Brunswick, of Pied Piper infamy), whose talk was entitled “Who Knows? AI-Infused Science — and what it means for science communication and education”. Her research has investigated how people are turning more and more towards LLM chatbots to answer scientific questions, when historically such questions would have been directed to seats of epistemic authority, such as academics, doctors, or journalists. I would be interested to know if the rise in chatbot use correlates with a measurable, corresponding decline in people seeking advice from their doctors, for instance.

Either way, the problem with this is obvious: LLMs make mistakes, and obscure the sources of the information they provide. Generally speaking, they amplify a consensus view, preferring to provide a concrete answer to the often messier, limited, or developing truth. Greussing highlighted the work of Bromme & Goldman (2014), who studied how laypeople decide what or whom to trust. They identified three pillars of trustworthiness: expertise, integrity, and benevolence. Assessing these qualities in a source allows the layperson to have informed trust in the information that source is providing. Greussing pointed out that, with LLMs taking a larger role in informing the public understanding of science, the conditions for informed trust may also be changing, and hence the tasks of science communication and education may have to change as well.

During the questions for Greussing, Guido Sanguinetti, one of the invited panel members for the final discussion which I shall shortly describe, made an interesting comparison between journalists and academics. He stated that journalists ‘sold out’ their profession by allowing news media to be acquired and controlled by large companies, which has led to general distrust of said media. He posited that AI may have the same consequence for academics; if we sell out by allowing large companies to become the (almost certainly) non-benevolent guardians of knowledge, and its sole distributors via AI, it will generate public distrust in traditional academic institutions.

Following Greussing’s talk, there was a panel discussion moderated by Roberto Trotta. The panellists were Guido Sanguinetti, Professor of Machine Learning and Systems Biology at SISSA; Emily Sullivan, Senior Lecturer in Philosophy at the University of Edinburgh; and Juan Carlos de Martin, Professor of Computer Engineering at the Polytechnic University of Turin. The topic of the discussion was “What future for human scientists?”.

The panel began with each of the panellists describing their thoughts on agentic AI in their fields. Sullivan explained that, in philosophy, the written report is the work, meaning that if the writing of a paper is farmed off to an agent, then the very thought process that the philosopher is meant to be doing has also been outsourced. She stated that ‘language is thought’ — which resonated with my own thoughts. I would also go one further than Sullivan, and state that the written report is the work in all fields, even in natural sciences. This is also David Hogg’s argument in his essay “Why do we do astrophysics?” — that astrophysics is the literature. The use of agentic AI therefore seems to be antithetical to the normal understanding of what it means to do research.

Other concerns highlighted by Sullivan were how we as academics are willingly giving away our most valuable asset, intellectual property, to big technology firms, via the prompting of LLMs and agents, and the potential top-down influence of said firms on the direction of science. Research questions will be easily re-directed, or silenced altogether, if agentic AI owned by private companies becomes the sole mode of research. Furthermore, by relying on a single source of information — an LLM — in the research process, we become epistemically vulnerable and unable to access out-of-the-box thinking.

De Martin spoke next, and posed the question that he said must be asked of any new technology: what problem is this tool trying to solve? What motivation does the company selling it to you have? Technology is ‘a human artefact’, resulting from choices, which we must question — and usually the answer to its reason for existence will relate more to profitability or power than benevolence. He also pointed out that a good deal of the data that LLMs are trained on is already public knowledge, which we are paying to access through these tools. Is this the first serious attempt to privatise research?

Sanguinetti pointed out that, due to the nature of the training data, the more specialised the question you pose to an LLM is, the less likely the answer is to be correct. The only situation in which it is reasonable to use an LLM is therefore when your question is not ‘mission critical’, and when the answer is easy and fast to verify.

A consensus having been reached that agentic AI is not a force for good in research, the discussion then moved on to how we can prevent a future in which scientists are nothing more than prompt engineers. It was noted that the narrative of inevitability — that AI is the future, whether you like it or not — serves the interests of the tech firms, and, of course, their investors. It seems that the only way we as scientists can resist this is collectively, but de Martin pointed out that this is difficult in a world where academia is increasingly depoliticised. The modern-day academic is hyper-individualistic — individual merits and successes are rewarded in academia — which is precisely what AI also promises to accelerate: the productivity of the individual. Changing this means changing the entire reward structure of global academia, which is then a question of politics rather than simple collective action. We do not have an easy road ahead of us.

The final quick-fire question posed to the panel was this: what are your reasons for optimism? Sullivan said that she is hopeful because people get into science because they value knowledge — as long as that does not change, agentic research will be resisted. De Martin said that the fact that the future is unknown and unknowable gives him hope that change can be effected. Sanguinetti said that he is optimistic because we are already talking about the problem, and will continue to do so.

I share his sentiment. Preparing my talk and attending this workshop have helped me to clarify why I find agentic research distasteful, despite participating in it myself for the past few months. I am seriously starting to consider giving up on using LLMs completely — and it actually feels a little ridiculous that this is something that requires serious consideration. I managed all of my research up until October 2025 without an LLM; I think I will be fine. I am not sorry that I used them, though. Without actually using these tools — without feeling the sense of amazement when first seeing them work, then the tug of addiction as my capability horizon appeared to instantly become infinite, and the twinge of frustration when the rate limit hit — I would not have been able to enter into this discussion to the extent that I have.

My reasons for optimism are these: that endless growth is a fallacy, and the tech companies behind the popular LLMs will eventually reach a ceiling of userbase and influence over science, and then decline. That LLMs will eventually become trained on so much LLM-generated text that they become unusable. That researchers will realise that without friction, without discomfort, without the satisfaction of a sore head when that brick wall finally crumbles, the work has no meaning. That we will claim the right to be unhappy.