Hello World with LangChain4j
Let's start with the simplest possible example — sending a message to an LLM and getting a response. LangChain4j provides the ChatLanguageModel interface that abstracts away the complexity of HTTP calls, authentication, and response parsing. This unified interface works consistently across different LLM providers, so you can switch from OpenAI to Anthropic or Google Gemini with minimal code changes.
The beauty of LangChain4j is that it follows familiar Java patterns. If you've used builders in Spring or Lombok, you'll feel right at home. The library handles connection pooling, retry logic, rate limiting, and error handling behind the scenes, letting you focus on building features rather than infrastructure.
Your First Chat Application
Here's the complete code for a minimal LangChain4j application that connects to OpenAI. This example demonstrates the three core steps you'll use in every LangChain4j application: configure the model, send a prompt, and process the response.
importdev.langchain4j.model.chat.ChatLanguageModel;importdev.langchain4j.model.openai.OpenAiChatModel;publicclassSimpleChatApp{publicstaticvoidmain(String[] args){// 1. Create the chat model (connect to OpenAI)ChatLanguageModel model =OpenAiChatModel.builder().apiKey(System.getenv("OPENAI_API_KEY")).modelName("gpt-4o-mini").temperature(0.7).build();// 2. Send a message and get a responseString response = model.generate("Explain Java virtual threads in 2 sentences");// 3. Print the responseSystem.out.println(response);}}That's It!
In just 10 lines of code, you've created an AI-powered Java application. No REST clients, no JSON parsing, no authentication headers — LangChain4j handles everything.
Understanding the Flow
Configure
Set API key, model name, and parameters
Generate
Send your prompt to the LLM
Receive
Get the response as a String
Prompt Engineering in Java
Prompt engineering is the art of crafting instructions that get the best results from LLMs. In LangChain4j, you have fine-grained control over how prompts are structured using system messages, user messages, and prompt templates. The difference between a mediocre AI application and an excellent one often comes down to how well your prompts are designed.
Think of prompt engineering as programming in natural language. Just like code, prompts need to be clear, unambiguous, and well-structured. Vague instructions lead to unpredictable outputs, while precise instructions yield consistent, high-quality results. LangChain4j gives you the tools to build robust, maintainable prompt systems.
System vs User Messages
LLMs distinguish between system messages (which set behavior and context) and user messages (which contain the actual request). This separation is crucial for consistent AI behavior. The system message acts like a configuration file for the AI's personality and capabilities, while user messages are the runtime inputs.
A well-crafted system message can transform a generic chatbot into a specialized expert. For example, you might configure the AI to respond only in JSON format, to always include code examples, or to avoid certain topics entirely. This is where enterprise applications gain their competitive advantage.
importdev.langchain4j.data.message.SystemMessage;importdev.langchain4j.data.message.UserMessage;importdev.langchain4j.model.chat.ChatLanguageModel;importdev.langchain4j.model.openai.OpenAiChatModel;publicclassSystemAndUserMessages{publicstaticvoidmain(String[] args){ChatLanguageModel model =OpenAiChatModel.builder().apiKey(System.getenv("OPENAI_API_KEY")).modelName("gpt-4o-mini").build();// System message: Sets the AI's personality and behaviorSystemMessage systemMessage =SystemMessage.from("You are a senior Java developer with 15 years of experience. "+"You give concise, practical advice with code examples. "+"You prefer modern Java features (17+) and design patterns.");// User message: The actual questionUserMessage userMessage =UserMessage.from("How should I handle null values in method parameters?");// Send both messagesString response = model.generate(systemMessage, userMessage).content().text();System.out.println(response);}}Pro Tip: System Message Best Practices
- • Be specific about the persona and expertise level
- • Define the output format you expect
- • Include constraints (e.g., "respond in under 100 words")
- • Mention what to avoid (e.g., "don't use deprecated APIs")
Prompt Templates with Variables
Hardcoding prompts isn't practical. LangChain4j provides powerful templating with variable substitution:
importdev.langchain4j.model.input.Prompt;importdev.langchain4j.model.input.PromptTemplate;publicclassPromptTemplates{publicstaticvoidmain(String[] args){// Define a template with placeholdersPromptTemplate template =PromptTemplate.from("You are an expert in {{language}}. "+"Explain the concept of {{concept}} to a {{level}} developer. "+"Use practical examples and keep it under {{maxWords}} words.");// Create a prompt by filling in variablesPrompt prompt = template.apply(Map.of("language","Java","concept","Dependency Injection","level","junior","maxWords","150"));// Use the promptString response = model.generate(prompt.text());System.out.println(response);}}Multi-line Prompt Templates
For complex prompts, use Java text blocks for better readability:
String promptTemplate ="""
You are a code reviewer for a Java project.
## Code to Review:
{{code}}
## Review Criteria:
1. Code quality and readability
2. Potential bugs or security issues
3. Performance concerns
4. Adherence to Java best practices
## Output Format:
Provide a structured review with:
- Overall score (1-10)
- Key issues (bullet points)
- Suggested improvements (with code examples)
""";PromptTemplate template =PromptTemplate.from(promptTemplate);Prompt prompt = template.apply(Map.of("code", userProvidedCode));Structured Outputs & POJO Mapping
Getting plain text from an LLM is useful, but in real applications you often need structured data. LangChain4j excels at extracting structured information and mapping LLM responses directly to Java objects. This is where the power of type-safe Java really shines — you get compile-time safety and IDE autocompletion for AI-generated data.
Consider common enterprise use cases: extracting customer information from emails, parsing invoice data from documents, or converting natural language queries into database filters. All of these require structured output that integrates seamlessly with your existing Java codebase. LangChain4j makes this not just possible, but elegant.
JSON Mode
The first step to structured output is enabling JSON mode. This forces the LLM to respond with valid JSON instead of free-form text, eliminating parsing errors and making the output predictable:
importdev.langchain4j.model.openai.OpenAiChatModel;importdev.langchain4j.model.openai.OpenAiChatModelName;ChatLanguageModel model =OpenAiChatModel.builder().apiKey(System.getenv("OPENAI_API_KEY")).modelName(OpenAiChatModelName.GPT_4_O_MINI).responseFormat("json_object")// Enable JSON mode.build();String response = model.generate("""
Extract the following information from this text and return as JSON:
Text: "John Smith, age 32, works as a Software Engineer at Google.
He has 8 years of experience and specializes in distributed systems."
Required fields: name, age, company, role, yearsOfExperience, specialization
""");// Response will be valid JSON:// {"name": "John Smith", "age": 32, "company": "Google", ...}Automatic POJO Extraction
The real power comes from automatic mapping to Java classes using AI Services:
importdev.langchain4j.service.AiServices;importdev.langchain4j.service.UserMessage;// 1. Define a POJO for the structured datarecordPerson(String name,int age,String company,String role,int yearsOfExperience,List<String> skills
){}// 2. Define an interface with extraction methodinterfacePersonExtractor{@UserMessage("Extract person information from: {{text}}")PersonextractPerson(@V("text")String text);}// 3. Create and use the servicePersonExtractor extractor =AiServices.create(PersonExtractor.class, model);Person person = extractor.extractPerson("Jane Doe is a 28-year-old Data Scientist at Netflix "+"with 5 years of experience in Python and Machine Learning.");// person.name() returns "Jane Doe"// person.age() returns 28// person.skills() returns ["Python", "Machine Learning"]Handling Extraction Failures
LLMs aren't perfect. Sometimes extraction fails or returns unexpected values. Always validate extracted data and have fallback strategies. Consider using Optional<T> for fields that might not be present in the source text.
Extracting Lists and Complex Types
// Extract a list of itemsinterfaceProductExtractor{@UserMessage("Extract all products mentioned in: {{text}}")List<Product>extractProducts(@V("text")String text);}recordProduct(String name,double price,String category){}// UsageList<Product> products = extractor.extractProducts("Our store sells iPhone 15 Pro at $999, MacBook Air at $1199, "+"and AirPods Pro at $249.");// Returns a list with 3 Product objects, each properly populated!Model Configuration Options
LangChain4j gives you fine-grained control over how the LLM behaves. Understanding these parameters helps you tune responses for your specific use case. The difference between a chatbot that feels slow and unreliable versus one that feels snappy and intelligent often comes down to these configuration choices.
Temperature is perhaps the most important parameter. Use low values (0.0-0.3) for factual tasks like data extraction or code generation where consistency matters. Use higher values (0.7-1.0) for creative tasks like brainstorming or writing where variety is desirable. Finding the right balance for your application requires experimentation, but LangChain4j makes it easy to A/B test different configurations.
| Parameter | Range | Effect |
|---|---|---|
temperature | 0.0 - 2.0 | Higher = more creative/random, Lower = more focused/deterministic |
topP | 0.0 - 1.0 | Nucleus sampling - controls diversity of token selection |
maxTokens | 1 - model max | Maximum length of the response |
timeout | Duration | How long to wait for a response |
ChatLanguageModel model =OpenAiChatModel.builder().apiKey(System.getenv("OPENAI_API_KEY")).modelName("gpt-4o-mini")// Creativity settings.temperature(0.3)// Low: factual responses.topP(0.9)// Slight diversity in word choice// Response limits.maxTokens(500)// Cap response length.timeout(Duration.ofSeconds(30))// Retry configuration.maxRetries(3).logRequests(true)// Debug logging.logResponses(true).build();🎉 Level 2 Complete!
You've mastered chat models and prompt engineering! You can now send messages to LLMs, structure prompts effectively, and extract structured data into Java objects. In the next level, we'll tackle memory and conversations — making your AI apps remember context across multiple interactions.