Subscribe now

Not only is it is open to the scientific community*, but it costs zero dollars per token. AND it hosts LLM, VLM, and embedding inference endpoints that live on dedicated nodes that avoid HPC queues and auto-scale. It also provides both API and OpenWebUI (open source ChatGPT) experiences. Since it relies on Globus Auth, many faculty are probably already technically authenticated to use it with an approved allocation2.

*The Catch

Ok so there is a little bit of friction and cannot just log in and start blasting through tokens. You need an allocation first. However, they do have a Director’s Discretionary (DD) Allocation designed for “startups” and getting code running. It doesn’t require DOE sponsorship. It’s the “let me just try this out” path of least resistance before applying for major awards.

Share

Once your DD proposal is approved, you will receive the Project Short Name and Principal Investigator (PI)details necessary to finally complete the account registration process.

The account activation and requirements FAQs are helpful, but if you want to skip to the Getting Started portal, the documentation is fairly robust and straightforward: https://www.alcf.anl.gov/get-started.

Here is what you will need for the form:

• Principal Investigator (PI) Name: The name of the project lead who will act as your “Sponsor” and approve your request.

• Project / Allocation: You have to apply and be approved for this before the account is created.

• Institutional Email: Use your .edu or .gov address (not Gmail/Yahoo) to avoid delays.

• ORCID iD: You are required to link your ORCID account, so ensure you have your login credentials ready.

• Legal Name & Citizenship info: You must accurately declare your citizenship status (passport)

• Curriculum Vitae (CV): A PDF copy of your CV is often required to be uploaded directly during this step. (for non-US Citizens)

• Token Preference: You will need to choose between a Mobile Token (app-based) or a Physical Cryptocard. - kind of like Duo.

More about the service

They are running this service on their Sophia cluster (24 nodes of NVIDIA DGX A100s) and the Metis cluster (SambaNova SN40L systems).

• Sophia uses vLLM and supports the full range of OpenAI-compatible endpoints including chat, completions, embeddings, and batch processing. (available models)

• Metis uses SambaNova’s inference API and currently supports only chat completions. (available models)

• There are two more clusters coming soon...possibly Solstice and Equinox….which feature tens do thousands of NVIDIA Blackwell GPUs.3

As stated earlier, there is a web interface (based on OpenWebUI) that looks like ChatGPT at https://inference.alcf.anl.gov. It provides multi-model chat access, RAG. agent capability, and all that you would expect from an AI chat interface these days. But the real fun and power is the programmatic access. The API is OpenAI-compliant. That means your researchers don’t need to rewrite code….just change the base_url in their Python script. So you could easily use it alongside the code you might have written with your on campus Portkey, LiteLLM, OpenRouter, or campus approved AI cloud APIs.

from openai import OpenAI
from inference_auth_token import get_access_token

# Get your access token
access_token = get_access_token()

client = OpenAI(
    api_key=access_token,
    base_url="https://inference-api.alcf.anl.gov/resource_server/metis/api/v1"
)

response = client.chat.completions.create(
    model="gpt-oss-120b-131072",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms."}]
)

print(response.choices[0].message.content)

Share

And this is just the tip of the iceberg, these inference services join all the other services and GPU power on the clusters. Some interesting examples they highlighted from the webinar:

1. Agentic Workflows: They showed off some real work with ChemGraph. This is an agentic workflow where the LLM acts as a router, calling out to Python libraries like RDKit to actually simulate molecular properties4.

2. AuroraGPT (site): This is a model fine-tuned on scientific literature.

3. Batch vs Real-Time inference: The inference endpoints are not just for chat and real time completions. Argonne has built a Batch API specifically for use cases when you want to dump a file with 150,000 requests and let their A100s chew through it.

4. All the other services: LLMs are only a small fraction of what these clusters offer for science. And some exciting upgrades are planned over the next few years due to public-private partnerships.

As with any external compute service, please check with your IT, research computing, and research offices first (they may want to check out the data policies and software policies to determine fit). But for researchers looking to balance the high demand for AI compute with flat budgets and have a project to lean on, the ALCF service could represent a significant opportunity.

For questions or support, please contact ALCF Support.

Subscribe now

And yeah, the numbers are bad. Unemployment for new bachelor’s degree holders is up over 6%….ugly compared to where we were a few years ago. The story goes that ChatGPT, Claude Code, Codex, and Cursor (what’s with all the C’s?) and the like are eating that entry-level work. Companies don’t need juniors anymore, so they say.

Subscribe now

I get why that narrative sticks. It’s scary, sensational, dramatic, and feels right to those of us who have experienced what these tools do. But in Glenda Morgan’s recent breakdown of this, she points out something inconvenient: the hiring slump actually started before the generative AI boom really took hold.

“But the argument is also, if not wrong, at least premature. Among other issues, the falloff in new graduate hiring predates the widespread adoption of generative AI. Despite these problems, it is an account that has gained a lot of traction. Because of this, there is a real danger that efforts to address the issue will be focused on the wrong thing.” - Glenda Morgan

EDIT: Oh and then there is this timely published study from 12/28 → In an end-of-year note to investors, Vanguard set the record straight: careers that research claims are most susceptible to AI are actually thriving, not dying out.

“The approximately 100 occupations most exposed to AI automation are actually outperforming the rest of the labor market in terms of job growth and real wage increases,” the Vanguard report revealed. “This suggests that current AI systems are generally enhancing worker productivity and shifting workers’ tasks toward higher-value activities.” - Emma Burleigh

This matters in higher ed because there already exists a climate of panic around the demographic cliff and worsening sentiment around higher ed ROI. If an AI bogeyman strikes fear into boards and administration, we may prematurely spend a fortune trying to solve AI. We’ll rewrite curriculums, buy a bunch of shiny new "AI readiness" platforms, engage in expensive Deloitte AI strategy and governance, and blindly give Microsoft even more money for more lame copilots. But if the real issue is just a crappy economy or high interest rates, none of that expensive work is going to change the hiring numbers.

It highlights something we’re terrible at in higher ed: we don’t actually know what’s happening with our own data.

University data is all over the place. Career Services has some of it, Alumni Relations has some, the Registrar has some, definitions and access rules are opaque and inconsistent, and quality is often an afterthought. It’s a freaking mess.

(Let’s not talk about data loss, privacy, and cybersecurity implications…)

So while everyone is wringing their hands about the "future of work," I’m over here still thinking about something much more boring: integration and data enablement. Are we going to make huge strategic bets based on anecdotes without a clear view of our pipeline? Are employers rejecting our students because they can’t use AI or because the market competition is flooded?

“There is a real danger that if we focus on the second phase (the expertise upheaval) rather than the first phase and the more immediate causes of the graduate hiring slump, we will fail to address the issue or address it only partially.” - Glenda Martin

Maybe AI really will wipe out entry-level jobs. But are we trying to treat a patient without looking at the chart first? Before we turn the whole university upside down chasing the AI trend, we should probably figure out what our data is actually trying to tell us. Yes…I am speaking of that same boring need to build stronger data cultures and capabilities.

It’s not as exciting as an "AI Revolution," but it’s probably a lot more useful. Just think of where you would be now had you started 10 years ago.

The second paragraph in Google’s announcement on Accelerate Medical Research in Bigquery describes this new service in BigQuery:

It likely costs thousands of dollars in compute time and significant overhead to keep a vector column up to date on 35 ‘million-ish’ articles to facilitate semantic search.

But Google has already done it. You don’t have to front the cost to vectorize the library (and maintain it).

So I asked myself….”well if BigQuery can now be an MCP server, then can I plug it into Claude Code, OpenWebUI, TypingMind, an OpenAI GPT, or wherever”?

So off I went on this adventure fools errand. In my GCP account, I followed their "Getting Started" guide, copied their sample query, modified it to include research topics from one of my researchers and watched the magic happen.

Cool. It worked. BUT 115 GB SCANNED FOR ONE QUERY. That sounds expensive. The excitement waned.

For a single question. At BigQuery’s on-demand rate, that’s around 70c per query after free-tier. Run that a thousand times for a serious research project, and it wouldn’t take much to watch those precious research credits go up in smoke. Stacking that on top of agent costs likely won’t make leadership happy and optimistic about the value of AI.

What I found after research is that the difference between a pocket-emptying query and a practically free one came down to deleting a single line of code.

The Trap of the "Easy" Query

The problem lies in how BigQuery bills you. As it is mostly concerned about how data hungry you are, and the PubMed dataset is massive, the real weight is in the article_text column.

In the example Google provides, the SQL asks the database to perform a vector search to identify the most semantically similar articles to your query. But it also asks the database to return the full text of those articles in the same swoop.

This sounds efficient, right? It is not.

When I ran that standard query, BigQuery had to scan the embedding column to do the math, but because I requested SELECT base.article_text, it also decided to haul back that massive text column along for the ride. That’s what led to the 100+ GB of data.

The ~90% Discount

But if the vector search only needs the embeddings (the mathematical representation of the text) to figure out which articles match, does it really need the human-readable text until a human wants to dive deeper? Could that be the job of a second agent and some human in the loop process? Or a deterministic workflow?

I started by the exact same query and deleted one line: base.article_text and hit run. The results came back just as fast, giving me the IDs, titles, and authors of the most relevant papers. But the data processed dropped from 115 GB down to 13ish GB.

By simply not pulling the full text through the computational pipeline, the price of the query dropped from a dollar to a dime. Those savings will add up.

Another Way to Build This

If you actually need the text (and spoiler: you probably do), maybe don’t ask for it during the search.

First, use the lean, 13GB query to find the top X Article IDs. That costs a few cents. Then, run a second, tiny query to fetch the full text for analysis 10 IDs. Cheaper.

It could probably even get better if you copy the table or just the rows you need to your own project (a one-time or maybe scheduled pull cost of a few dollars) and build your own vector index. Then you stop scanning GB and start scanning MB.

The Lesson

Cloud providers love to show you the "happy path" code that gets you a result in five seconds. FinOPS batteries are often not included. Watch out.

(Again: Please correct my misunderstandings of BigQuery and help me learn).

Ok now on to the MCP/A2A part of my journey….