Optimizing AI Coding Assistants for Large Java Applications

When developers work with large, traditional Java applications (like Spring Boot, Servlets, or JSP), using AI coding assistants can become very expensive and slow. Every time you ask the AI a question, it reads a lot of code. This uses a large number of tokens (the words or pieces of code that AI reads), which increases your cloud bill.

To solve this problem, AI Copilot Optimizer acts as a smart middle layer between your code editor and OpenAI. It remembers previous questions, cleans your data for safety, and helps you save money.

You can find the open-source code here: github.com/sonani-pankaj/AI-Copilot-optimizer.

How It Works (Architecture)

The system has three main parts: your code editor (like VS Code or IntelliJ), a fast Python backend, and a visual dashboard to see your savings.

Here is how the data moves through the system:

%%{init: {
"themeVariables": {
  "fontSize": "12px"
},
"flowchart": {
  "padding": 10,
  "nodeSpacing": 30,
  "rankSpacing": 40
}
}}%%
graph TD
User[Code Editor<br/>VS Code / IntelliJ] -->|JWT, Query,<br/>Git Diff, Symbols| Backend[FastAPI Backend<br/>Python 3.12]

subgraph Backend_Process [Backend Components]
  Backend --> Sanitizer[1. Sanitizer<br/>Removes Secret Keys]
  Sanitizer --> Redis[2. Redis Cache<br/>24h Memory]
  Redis --> FAISS[3. FAISS Search<br/>Similar Questions]
  FAISS --> Postgres[4. Postgres<br/>Persistent Storage]
  Postgres --> OpenAI[5. OpenAI<br/>Fallback gpt-4o-mini]
  end

Dashboard[Vite + React<br/>Dashboard] -->|Reads Stats<br/>and Cache| Backend

Simple Explanation of the Components

Sanitizer: Before sending any code to the internet, this component checks and deletes private passwords or secret API keys.
Redis (Hot Cache): A super-fast temporary memory. If you ask the exact same question within 24 hours, it gives you the answer instantly without asking OpenAI.
FAISS (Vector Search): If your question is almost the same as an old one, FAISS recognizes the similarity and reuses the old answer.
Postgres: A permanent database that saves history and statistics.
OpenAI: The actual AI system (gpt-4o-mini). The project only talks to OpenAI if it cannot find an answer in Redis or FAISS.

Prerequisites

Here’s a beginner-friendly local setup tutorial for sonani-pankaj/AI-Copilot-optimizer that you can follow step by step.

This repo is a monorepo with:

a FastAPI backend on port 8000
a React dashboard on port 5173
Postgres on port 5432
Redis on port 6379
an Ollama local LLM on port 11434
optional editor integrations for VS Code and IntelliJ.

The setup details below are based on the repository’s own docs and config files, especially .github/AI_CONTEXT.md, DEVELOPER.md, README.md, .env.example, and docker-compose.yml.

What you need before you start

Install these tools first:

Tool	Recommended version
Git	latest
Python	3.12
Node.js	20+
Docker Desktop	recent version
Ollama	latest
VS Code	latest
IntelliJ IDEA	optional

The repo docs explicitly call out Python 3.12, Node 20+, Docker, and Ollama for local setup.

Clone the repository

Open a terminal and run:

git clone https://github.com/sonani-pankaj/AI-Copilot-optimizer.git
cd AI-Copilot-optimizer

If your folder name appears as AI-Copilot-optimizer, stay inside that root folder for most commands.

Understand what will run locally

This project is intended to run mostly on your own machine:

Backend: FastAPI at http://localhost:8000
Dashboard: Vite React app at http://localhost:5173
Postgres: localhost:5432
Redis: localhost:6379
Ollama: http://localhost:11434

Important note from the repo docs:

For development, you usually run Postgres + Redis in Docker
and run backend + dashboard natively on your machine for hot reload.

Create the `.env` file

In the project root, create a file named .env.

The repo already provides .env.example, and the project context also documents the required variables.

Easiest way

Copy the example file:

cp .env.example .env

If you are on Windows PowerShell and cp behaves differently, you can do:

Copy-Item .env.example .env

Fill the `.env` file correctly

Open .env and make sure it contains values like this:

JWT_SECRET=<your-generated-secret>
DATABASE_URL=postgresql+asyncpg://postgres:changeme@localhost:5432/ai_optimizer
REDIS_URL=redis://localhost:6379/0
OPENAI_API_KEY=ollama
OPENAI_BASE_URL=http://localhost:11434/v1
LLM_MODEL=gemma3:latest
POSTGRES_PASSWORD=changeme
DEMO_USERNAME=admin
DEMO_PASSWORD_HASH=<your-bcrypt-hash>

What each variable means

JWT_SECRET → used to sign login tokens
DATABASE_URL → backend connection to Postgres
REDIS_URL → backend connection to Redis
OPENAI_API_KEY=ollama → tells the OpenAI SDK to talk to Ollama locally
OPENAI_BASE_URL=http://localhost:11434/v1 → routes model requests to Ollama
LLM_MODEL=gemma3:latest → default local model
POSTGRES_PASSWORD=changeme → password used by Docker Postgres
DEMO_USERNAME=admin → login username
DEMO_PASSWORD_HASH → hashed password for login

Generate a secure `JWT_SECRET`

Run this command:

python -c "import secrets; print(secrets.token_hex(20))"

It will print a random 40-character hex string. Copy that and paste it into .env as JWT_SECRET.

Example:

JWT_SECRET=abc123...yourgeneratedvalue...

The app expects a bcrypt hash, not a plain password.

Run:

python -c "import bcrypt; print(bcrypt.hashpw(b'admin', bcrypt.gensalt(12)).decode())"

If Python says bcrypt is missing, install it first:

pip install bcrypt

Now copy the output hash into .env:

DEMO_PASSWORD_HASH=$2b$12$...

If you use the command above, the password will be:

username: admin
password: admin

That’s the simplest option for first-time setup.

Install the Ollama model

This project’s local docs say it uses Ollama by default, with model gemma3:latest.

Run:

ollama pull gemma3:latest

Then confirm:

ollama list

You should see gemma3:latest in the list.

If Ollama is not running

Start Ollama on your machine first. On most systems, installing and launching Ollama starts the service automatically.

Start Postgres and Redis with Docker

From the project root, run:

docker compose up -d postgres redis

Then check status:

docker compose ps

You should see both services running or healthy.

Why only these two?

The project docs recommend:

run Postgres and Redis in Docker
run backend and dashboard directly on your machine during development

That gives you easier debugging and hot reload.

Set up the backend Python environment

Go into the backend folder:

cd backend

Create a virtual environment:

python -m venv .venv

Activate it:

Windows

.venv\Scripts\activate

macOS/Linux

source .venv/bin/activate

Install dependencies:

pip install -r requirements.txt

Start the backend correctly

This is very important:

Run uvicorn from the project root, not from inside backend/.

The project docs explicitly warn that running from inside backend/ can break relative imports.

So first go back to the repo root:

cd ..

Now start the backend.

Windows

backend\.venv\Scripts\uvicorn.exe backend.main:app --reload --port 8000

macOS/Linux

backend/.venv/bin/uvicorn backend.main:app --reload --port 8000

If everything is correct, the backend should start on:

http://localhost:8000

Swagger docs should be available at:

http://localhost:8000/docs

Verify the backend is healthy

Open a browser and visit:

http://localhost:8000/health

You should get something like:

{"status":"ok"}

If yes, the backend is working.

You will need this token for the VS Code extension and possibly for testing API calls.

You can skip manual token generation if you only want to log in via the dashboard first.

Option B: Generate the token now from terminal

PowerShell

$token = (Invoke-WebRequest -Uri http://localhost:8000/auth/login -Method POST `
  -ContentType "application/json" `
  -Body '{"username":"admin","password":"admin"}' `
  -UseBasicParsing | ConvertFrom-Json).access_token
Write-Host $token

curl

curl -X POST "http://localhost:8000/auth/login" \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"admin"}'

Copy the access_token and save it somewhere temporary.

Start the dashboard

Open a new terminal, go to the project root, then:

cd dashboard
npm install
npm run dev

This starts the dashboard at:

http://localhost:5173

Open that in your browser.

username: admin
password: admin

If your bcrypt hash was generated from a different password, use that password instead.

What you should now have running

At this point, these should be active:

Postgres via Docker
Redis via Docker
Ollama locally
FastAPI backend on port 8000
React dashboard on port 5173

That is the core local development setup.

Optional: Run everything with Docker instead

The repository’s docker-compose.yml also defines:

backend
dashboard

So if you want a containerized full stack, you can run:

docker compose up -d

This will start:

postgres
redis
backend
dashboard

Mapped ports from the compose file are:

5432:5432
6379:6379
8000:8000
5173:80

But for day-to-day development, the repo docs still recommend native backend/dashboard plus Docker for DB/cache.

Install and test the VS Code extension

If you want to use the VS Code extension, do this next.

Go to the extension folder:

cd vscode-extension
npm install
npm run compile

Package the extension:

npx @vscode/vsce package --no-dependencies

This creates a .vsix file, typically something like:

ai-copilot-optimizer-1.0.0.vsix

Install it:

code --install-extension ai-copilot-optimizer-1.0.0.vsix

Reload VS Code if prompted.

Connect the VS Code extension to the backend

Open VS Code.

Then:

Press Ctrl+Shift+P
Search for: AI Copilot Optimizer: Set Backend JWT Token
Paste the JWT token you generated earlier

Now open a Java project and try one of these:

open a .java file and save it
run Build Optimized Copilot Prompt
right-click a Java folder and use Build Optimized Copilot Prompt (Folder)

According to the repo docs, the extension sends:

JWT
query
git diff
symbols

to the backend.

Optional: Use VS Code extension in development mode

If you want to develop the extension itself:

Open the vscode-extension/ folder as the workspace root in VS Code
Press F5

A new Extension Development Host window will open.

The repo docs mention that F5 depends on the extension folder being opened correctly and having its .vscode/launch.json and tasks.json.

Optional: Run the IntelliJ plugin

If you use IntelliJ IDEA, you can also run the plugin.

Go to:

cd intellij-plugin

Run:

Windows

.\gradlew.bat runIde

macOS/Linux

./gradlew runIde

A sandbox IntelliJ instance should open with the plugin loaded.

Inside the sandbox IDE:

go to Settings
find AI Copilot Optimizer
configure backend URL and JWT token

Backend URL should usually be:

http://localhost:8000

Recommended startup order every day

For a fresher, this is the safest daily sequence:

Step A

Start Ollama and make sure the model exists:

ollama list

Step B

Start infra:

docker compose up -d postgres redis

Step C

Start backend from project root:

Windows

backend\.venv\Scripts\uvicorn.exe backend.main:app --reload --port 8000

macOS/Linux

backend/.venv/bin/uvicorn backend.main:app --reload --port 8000

Step C

Start backend from project root:

Windows

backend\.venv\Scripts\uvicorn.exe backend.main:app --reload --port 8000

macOS/Linux

backend/.venv/bin/uvicorn backend.main:app --reload --port 8000

Step D

Start dashboard:

cd dashboard
npm run dev

Step E

Open:

http://localhost:5173
http://localhost:8000/docs

First things to test after setup

After everything is up, test in this order:

Test 1: Backend health

Open:

http://localhost:8000/health

Expected:

{"status":"ok"}

Test 2: Dashboard loads

Open:

http://localhost:5173

Expected:

Use:

username: admin
password: admin

Expected:

dashboard opens

Test 4: API docs load

Open:

http://localhost:8000/docs

Expected:

Swagger UI page appears

Test 5: VS Code extension

Expected:

after setting JWT, it can call backend without auth errors

Common errors and how to fix them

Error: `JWT_SECRET env var is not set`

Cause: .env missing or backend started from wrong directory.

Fix:

make sure .env is in the project root
run uvicorn from the project root, not from backend/

Error: Postgres connection refused

Fix:

docker compose up -d postgres

Then check:

docker compose ps

Error: Redis connection refused

Fix:

docker compose up -d redis

Cause: password hash doesn’t match the password you are typing.

Fix: Regenerate the hash:

python -c "import bcrypt; print(bcrypt.hashpw(b'admin', bcrypt.gensalt(12)).decode())"

Put it into .env, restart backend, then try logging in again.

Error: `502 LLM unavailable`

Cause: Ollama is not running, or model name does not match.

Fix:

ollama list

Make sure .env contains the same model name, for example:

LLM_MODEL=gemma3:latest

If not installed:

ollama pull gemma3:latest

Error: Python cannot import `bcrypt`

Fix:

pip install bcrypt

Error: Extension shows no output

Fix checklist:

backend is running on port 8000
JWT token has been set in the extension
you are testing on a Java file/folder
the token has not expired

Error: bcrypt/passlib incompatibility

The repo docs explicitly warn that:

passlib 1.7.4 is incompatible with bcrypt >= 4
bcrypt==3.2.2 is pinned and should not be upgraded casually

So if auth hashing breaks, check backend dependencies and avoid upgrading bcrypt blindly.

Important beginner tips

Always keep one terminal for backend logs
Use a second terminal for dashboard
Use Docker only for Postgres and Redis during development
Do not run backend from inside backend/
Keep .env only in the project root
If something changes in .env, restart the backend
If login fails, recheck DEMO_PASSWORD_HASH
If AI responses fail, recheck Ollama + model name

Minimal quick-start version

If you just want the shortest path:

git clone https://github.com/sonani-pankaj/AI-Copilot-optimizer.git
cd AI-Copilot-optimizer
cp .env.example .env

Update .env with:

a real generated JWT_SECRET
a fresh DEMO_PASSWORD_HASH

Then run:

ollama pull gemma3:latest
docker compose up -d postgres redis
cd backend
python -m venv .venv

Activate venv and install deps:

pip install -r requirements.txt
cd ..
backend/.venv/bin/uvicorn backend.main:app --reload --port 8000

In another terminal:

cd dashboard
npm install
npm run dev

Then open:

http://localhost:5173
login with admin/admin

How It Works (Architecture)

Simple Explanation of the Components

Prerequisites

What you need before you start

Clone the repository

Understand what will run locally

Create the .env file

Easiest way

Fill the .env file correctly

What each variable means

Generate a secure JWT_SECRET

Generate the login password hash

Install the Ollama model

If Ollama is not running

Start Postgres and Redis with Docker

Why only these two?

Set up the backend Python environment

Windows

macOS/Linux

Start the backend correctly

Windows

macOS/Linux

Verify the backend is healthy

13) Get a JWT login token

Option A: Use the dashboard login later

Option B: Generate the token now from terminal

PowerShell

curl

Start the dashboard

What you should now have running

Optional: Run everything with Docker instead

Install and test the VS Code extension

Connect the VS Code extension to the backend

Optional: Use VS Code extension in development mode

Optional: Run the IntelliJ plugin

Windows

macOS/Linux

Recommended startup order every day

Step A

Step B

Step C

Windows

macOS/Linux

Step C

Windows

macOS/Linux

Step D

Step E

First things to test after setup

Test 1: Backend health

Test 2: Dashboard loads

Test 3: Login works

Test 4: API docs load

Test 5: VS Code extension

Common errors and how to fix them

Error: JWT_SECRET env var is not set

Error: Postgres connection refused

Error: Redis connection refused

Error: Dashboard login fails

Error: 502 LLM unavailable

Error: Python cannot import bcrypt

Error: Extension shows no output

Error: bcrypt/passlib incompatibility

Important beginner tips

Minimal quick-start version

Create the `.env` file

Fill the `.env` file correctly

Generate a secure `JWT_SECRET`

Error: `JWT_SECRET env var is not set`

Error: `502 LLM unavailable`

Error: Python cannot import `bcrypt`