Overview
Services translate high-level operations (like "search for documents") into concrete database calls and caching strategies. They enforce invariants, manage distributed transactions, and are easily unit-testable because they depend only on DAOs and caches, not HTTP.
Golden rule: Services call DAOs; handlers call services; nothing calls handlers except Gin.
Dependency Injection Pattern
Every service receives its dependencies in a constructor:
type ChatService struct {
dao *DAO
cache Cache
llmClient LLMClient
logger Logger
}
func NewChatService(dao *DAO, cache Cache, llm LLMClient, log Logger) *ChatService {
return &ChatService{dao: dao, cache: cache, llmClient: llm, logger: log}
}
func (s *ChatService) Chat(ctx context.Context, req *ChatRequest) (*ChatResponse, error) {
// uses s.dao, s.cache, s.llmClient
}This enables easy mocking in tests:
mockDAO := &MockDAO{} // implement DAO interface
mockCache := NewMemoryCache()
svc := NewChatService(mockDAO, mockCache, mockLLM, logger)
// Test runs without database or network calls
result, err := svc.Chat(ctx, &ChatRequest{...})Core Services
| Service | Responsibilities | Key Methods |
|---|---|---|
| ChatService | Conversation management, context, token budgeting | Chat, StreamChat, GetHistory, DeleteConversation |
| SearchService | Query expansion, ranking, deduplication | Search, SemanticSearch, RankedSearch, GetStats |
| KBService | Knowledge base lifecycle, document indexing | CreateKB, DeleteKB, AddDocument, GetStats |
| DocumentService | Document processing pipeline, chunking | Upload, Reprocess, Delete, GetChunks |
| LLMService | LLM provider abstraction, rate limiting | Chat, Embed, Tokenize, CheckQuota |
| EmbeddingService | Vector storage, similarity search | Store, Search, Upsert, Delete |
Transaction Management
Services coordinate multi-step operations as transactions:
func (s *KBService) AddDocument(ctx context.Context, kbID, docPath string) error {
tx := s.dao.db.BeginTx(ctx, nil)
// Step 1: Create document record
doc, err := s.dao.CreateDocument(tx, kbID, docPath)
if err != nil {
tx.Rollback()
return err
}
// Step 2: Process document (extract text)
chunks, err := s.processDocument(doc)
if err != nil {
tx.Rollback()
return err
}
// Step 3: Create chunks and embeddings
for _, chunk := range chunks {
err := s.dao.CreateChunk(tx, chunk)
if err != nil {
tx.Rollback()
return err
}
}
// Step 4: Update KB index
err = s.dao.UpdateKBStats(tx, kbID)
if err != nil {
tx.Rollback()
return err
}
tx.Commit()
return nil
}Caching Strategy
Embedding Cache
Services maintain an LRU cache of embedding vectors to avoid repeated API calls:
func (s *SearchService) Search(ctx context.Context, query string) ([]Result, error) {
// Try cache first
embedding := s.cache.Get(query)
if embedding == nil {
// Cache miss: call LLM
var err error
embedding, err = s.llmService.Embed(ctx, query)
if err != nil {
return nil, err
}
s.cache.Set(query, embedding)
}
// Use embedding to search vectors in database
results, err := s.dao.SearchVectors(ctx, embedding, topK=10)
return results, err
}KB Metadata Cache
Cache knowledge base metadata (document count, last updated) with TTL:
func (s *SearchService) GetKBStats(ctx context.Context, kbID string) (*KBStats, error) {
key := fmt.Sprintf("kb_stats:%s", kbID)
stats := s.cache.Get(key)
if stats != nil {
return stats, nil
}
stats, err := s.dao.GetKBStats(ctx, kbID)
if err != nil {
return nil, err
}
s.cache.SetWithTTL(key, stats, 5*time.Minute)
return stats, nil
}Error Handling & Retries
Services handle transient errors gracefully:
func (s *EmbeddingService) GenerateEmbedding(ctx context.Context, text string) ([]float32, error) {
var result []float32
var lastErr error
for attempt := 0; attempt < 3; attempt++ {
result, err := s.llmClient.Embed(ctx, text)
if err == nil {
return result, nil
}
lastErr = err
if !isTransient(err) {
return nil, err // Don't retry permanent errors
}
backoff := time.Duration(1<Quota & Rate Limiting
Services enforce user quotas:
func (s *SearchService) Search(ctx context.Context, userID string, query string) ([]Result, error) {
// Check quota
quota, err := s.dao.GetUserQuota(ctx, userID)
if err != nil {
return nil, err
}
if quota.SearchCount >= quota.SearchLimit {
return nil, fmt.Errorf("quota exceeded: %d/%d", quota.SearchCount, quota.SearchLimit)
}
// Execute search
results, err := s.doSearch(ctx, query)
if err != nil {
return nil, err
}
// Update quota
s.dao.IncrementSearchCount(ctx, userID)
return results, nil
}Submodules
Chat Service Implementation
Conversation state management, context window, token accounting, streaming.
Search Service Implementation
Query rewriting, vector similarity, result ranking, deduplication.
Knowledge Base Lifecycle
Create, delete, re-index KB. Document batch upload and processing.
Caching Patterns
LRU, TTL-based, cache invalidation strategies.
Transaction Boundaries
ACID guarantees, rollback on error, distributed transactions.
Testing Services
Mocking dependencies, table-driven tests, fuzz testing.
Key Concepts
Dependency Injection
Services receive all dependencies as constructor parameters, not as global singletons. This enables testing and makes dependencies explicit.
Interface-Based Design
Services depend on interfaces (DAO, Cache, Logger), not concrete types. This allows mock implementations in tests:
type DAO interface {
GetKnowledgeBase(ctx context.Context, id string) (*KB, error)
CreateDocument(ctx context.Context, doc *Document) error
SearchVectors(ctx context.Context, embedding []float32, k int) ([]Result, error)
}Context Propagation
Every method receives context.Context, enabling cancellation and timeout:
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
result, err := service.Search(ctx, query) // times out after 5s
Common Patterns
Option Functions (Builder Pattern)
Services can accept optional configurations:
type SearchOptions struct {
TopK int
Filter string
SortBy string
Timeout time.Duration
}
func (s *SearchService) SearchWithOptions(ctx context.Context, query string, opts *SearchOptions) ([]Result, error) {
if opts == nil {
opts = &SearchOptions{TopK: 10}
}
// use opts.TopK, opts.Filter, etc.
}Advanced Patterns
Circuit Breaker Pattern
If an external API (LLM, vector DB) fails repeatedly, stop calling it temporarily to avoid cascading failures.
type CircuitBreaker struct {
failures int
lastErr time.Time
state string // "closed", "open", "half-open"
}
func (cb *CircuitBreaker) Call(fn func() error) error {
if cb.state == "open" && time.Since(cb.lastErr) < 30*time.Second {
return fmt.Errorf("circuit open")
}
err := fn()
if err != nil {
cb.failures++
cb.lastErr = time.Now()
if cb.failures > 5 {
cb.state = "open"
}
return err
}
cb.failures = 0
cb.state = "closed"
return nil
}
Bulkhead Pattern
Isolate critical resources. For example, use a separate thread pool for LLM calls so they don't starve database queries.
type Services struct {
llmPool *WorkerPool // max 10 concurrent LLM calls
dbPool *WorkerPool // max 50 concurrent DB calls
}
func (s *Services) Embed(ctx context.Context, text string) ([]float32, error) {
result := make(chan []float32, 1)
err := s.llmPool.Submit(func() {
emb, _ := s.llmClient.Embed(ctx, text)
result <- emb
})
select {
case emb := <-result:
return emb, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
Observability Patterns
Every service method should emit metrics and logs for troubleshooting.
func (s *SearchService) Search(ctx context.Context, req *SearchRequest) (*SearchResponse, error) {
start := time.Now()
defer func() {
duration := time.Since(start)
s.metrics.SearchLatency.Observe(duration.Seconds())
s.logger.Info("search completed", "kb_id", req.KnowledgeBaseID, "ms", duration.Milliseconds())
}()
// ... search logic ...
}
Further Reading
- Handler Layer — Calls services
- DAO Layer — Called by services
- Internal Module Overview