backend-patterns

--- name: backend-patterns description: Backend architecture patterns, API design, database optimization, and server-side best practices for Node.js, Express, and Next.js API routes. metadata: origin: ECC --- # Backend Development Patterns Backend architecture patterns and best practices for scalable server-side applications. ## When to Activate - Designing REST or GraphQL API endpoints - Implementing repository, service, or controller layers - Optimizing database queries (N+1, indexing, connection pooling) - Adding caching (Redis, in-memory, HTTP cache headers) - Setting up background jobs or async processing - Structuring error handling and validation for APIs - Building middleware (auth, logging, rate limiting) ## API Design Patterns ### RESTful API Structure ```typescript // PASS: Resource-based URLs GET /api/markets # List resources GET /api/markets/:id # Get single resource POST /api/markets # Create resource PUT /api/markets/:id # Replace resource PATCH /api/markets/:id # Update resource DELETE /api/markets/:id # Delete resource // PASS: Query parameters for filtering, sorting, pagination GET /api/markets?status=active&sort=volume&limit=20&offset=0 ``` ### Repository Pattern ```typescript // Abstract data access logic interface MarketRepository { findAll(filters?: MarketFilters): Promise<Market[]> findById(id: string): Promise<Market | null> create(data: CreateMarketDto): Promise<Market> update(id: string, data: UpdateMarketDto): Promise<Market> delete(id: string): Promise<void> } class SupabaseMarketRepository implements MarketRepository { async findAll(filters?: MarketFilters): Promise<Market[]> { let query = supabase.from('markets').select('*') if (filters?.status) { query = query.eq('status', filters.status) } if (filters?.limit) { query = query.limit(filters.limit) } const { data, error } = await query if (error) throw new Error(error.message) return data } // Other methods... } ``` ### Service Layer Pattern ```typescript // Business logic separated from data access class MarketService { constructor(private marketRepo: MarketRepository) {} async searchMarkets(query: string, limit: number = 10): Promise<Market[]> { // Business logic const embedding = await generateEmbedding(query) const results = await this.vectorSearch(embedding, limit) // Fetch full data const markets = await this.marketRepo.findByIds(results.map(r => r.id)) // Sort by similarity return markets.sort((a, b) => { const scoreA = results.find(r => r.id === a.id)?.score || 0 const scoreB = results.find(r => r.id === b.id)?.score || 0 return scoreA - scoreB }) } private async vectorSearch(embedding: number[], limit: number) { // Vector search implementation } } ``` ### Middleware Pattern ```typescript // Request/response processing pipeline export function withAuth(handler: NextApiHandler): NextApiHandler { return async (req, res) => { const token = req.headers.authorization?.replace('Bearer ', '') if (!token) { return res.status(401).json({ error: 'Unauthorized' }) } try { const user = await verifyToken(token) req.user = user return handler(req, res) } catch (error) { return res.status(401).json({ error: 'Invalid token' }) } } } // Usage export default withAuth(async (req, res) => { // Handler has access to req.user }) ``` ## Database Patterns ### Query Optimization ```typescript // PASS: GOOD: Select only needed columns const { data } = await supabase .from('markets') .select('id, name, status, volume') .eq('status', 'active') .order('volume', { ascending: false }) .limit(10) // FAIL: BAD: Select everything const { data } = await supabase .from('markets') .select('*') ``` ### N+1 Query Prevention ```typescript // FAIL: BAD: N+1 query problem const markets = await getMarkets() for (const market of markets) { market.creator = await getUser(market.creator_id) // N queries } // PASS: GOOD: Batch fetch const markets = await getMarkets() const creatorIds = markets.map(m => m.creator_id) const creators = await getUsers(creatorIds) // 1 query const creatorMap = new Map(creators.map(c => [c.id, c])) markets.forEach(market => { market.creator = creatorMap.get(market.creator_id) }) ``` ### Transaction Pattern ```typescript async function createMarketWithPosition( marketData: CreateMarketDto, positionData: CreatePositionDto ) { // Use Supabase transaction const { data, error } = await supabase.rpc('create_market_with_position', { market_data: marketData, position_data: positionData }) if (error) throw new Error('Transaction failed') return data } // SQL function in Supabase CREATE OR REPLACE FUNCTION create_market_with_position( market_data jsonb, position_data jsonb ) RETURNS jsonb LANGUAGE plpgsql AS $$ BEGIN -- Start transaction automatically INSERT INTO markets VALUES (market_data); INSERT INTO positions VALUES (position_data); RETURN jsonb_build_object('success', true); EXCEPTION WHEN OTHERS THEN -- Rollback happens automatically RETURN jsonb_build_object('success', false, 'error', SQLERRM); END; $$; ``` ## Caching Strategies ### Redis Caching Layer ```typescript class CachedMarketRepository implements MarketRepository { constructor( private baseRepo: MarketRepository, private redis: RedisClient ) {} async findById(id: string): Promise<Market | null> { // Check cache first const cached = await this.redis.get(`market:${id}`) if (cached) { return JSON.parse(cached) } // Cache miss - fetch from database const market = await this.baseRepo.findById(id) if (market) { // Cache for 5 minutes await this.redis.setex(`market:${id}`, 300, JSON.stringify(market)) } return market } async invalidateCache(id: string): Promise<void> { await this.redis.del(`market:${id}`) } } ``` ### Cache-Aside Pattern ```typescript async function getMarketWithCache(id: string): Promise<Market> { const cacheKey = `market:${id}` // Try cache const cached = await redis.get(cacheKey) if (cached) return JSON.parse(cached) // Cache miss - fetch from DB const market = await db.markets.findUnique({ where: { id } }) if (!market) throw new Error('Market not found') // Update cache await redis.setex(cacheKey, 300, JSON.stringify(market)) return market } ``` ## Error Handling Patterns ### Centralized Error Handler ```typescript class ApiError extends Error { constructor( public statusCode: number, public message: string, public isOperational = true ) { super(message) Object.setPrototypeOf(this, ApiError.prototype) } } export function errorHandler(error: unknown, req: Request): Response { if (error instanceof ApiError) { return NextResponse.json({ success: false, error: error.message }, { status: error.statusCode }) } if (error instanceof z.ZodError) { return NextResponse.json({ success: false, error: 'Validation failed', details: error.issues }, { status: 400 }) } // Log unexpected errors console.error('Unexpected error:', error) return NextResponse.json({ success: false, error: 'Internal server error' }, { status: 500 }) } // Usage export async function GET(request: Request) { try { const data = await fetchData() return NextResponse.json({ success: true, data }) } catch (error) { return errorHandler(error, request) } } ``` ### Retry with Exponential Backoff ```typescript async function fetchWithRetry<T>( fn: () => Promise<T>, maxRetries = 3 ): Promise<T> { let lastError: Error for (let i = 0; i < maxRetries; i++) { try { return await fn() } catch (error) { lastError = error as Error if (i < maxRetries - 1) { // Exponential backoff: 1s, 2s, 4s const delay = Math.pow(2, i) * 1000 await new Promise(resolve => setTimeout(resolve, delay)) } } } throw lastError! } // Usage const data = await fetchWithRetry(() => fetchFromAPI()) ``` ## Authentication & Authorization ### JWT Token Validation ```typescript import jwt from 'jsonwebtoken' interface JWTPayload { userId: string email: string role: 'admin' | 'user' } export function verifyToken(token: string): JWTPayload { try { const payload = jwt.verify(token, process.env.JWT_SECRET!) as JWTPayload return payload } catch (error) { throw new ApiError(401, 'Invalid token') } } export async function requireAuth(request: Request) { const token = request.headers.get('authorization')?.replace('Bearer ', '') if (!token) { throw new ApiError(401, 'Missing authorization token') } return verifyToken(token) } // Usage in API route export async function GET(request: Request) { const user = await requireAuth(request) const data = await getDataForUser(user.userId) return NextResponse.json({ success: true, data }) } ``` ### Role-Based Access Control ```typescript type Permission = 'read' | 'write' | 'delete' | 'admin' interface User { id: string role: 'admin' | 'moderator' | 'user' } const rolePermissions: Record<User['role'], Permission[]> = { admin: ['read', 'write', 'delete', 'admin'], moderator: ['read', 'write', 'delete'], user: ['read', 'write'] } export function hasPermission(user: User, permission: Permission): boolean { return rolePermissions[user.role].includes(permission) } export function requirePermission(permission: Permission) { return (handler: (request: Request, user: User) => Promise<Response>) => { return async (request: Request) => { const user = await requireAuth(request) if (!hasPermission(user, permission)) { throw new ApiError(403, 'Insufficient permissions') } return handler(request, user) } } } // Usage - HOF wraps the handler export const DELETE = requirePermission('delete')( async (request: Request, user: User) => { // Handler receives authenticated user with verified permission return new Response('Deleted', { status: 200 }) } ) ``` ## Rate Limiting Rate limiting must use a shared store such as Redis, a gateway, or the platform's native limiter. Do not use per-process in-memory counters for production APIs: they reset on deploy, split across replicas, and fail open in serverless or multi-instance environments. Keep the backend layer responsible for choosing the integration point and error shape; use `api-design` for the HTTP contract and `security-review` for abuse case review. ## Background Jobs & Queues ### Simple Queue Pattern ```typescript class JobQueue<T> { private queue: T[] = [] private processing = false async add(job: T): Promise<void> { this.queue.push(job) if (!this.processing) { this.process() } } private async process(): Promise<void> { this.processing = true while (this.queue.length > 0) { const job = this.queue.shift()! try { await this.execute(job) } catch (error) { console.error('Job failed:', error) } } this.processing = false } private async execute(job: T): Promise<void> { // Job execution logic } } // Usage for indexing markets interface IndexJob { marketId: string } const indexQueue = new JobQueue<IndexJob>() export async function POST(request: Request) { const { marketId } = await request.json() // Add to queue instead of blocking await indexQueue.add({ marketId }) return NextResponse.json({ success: true, message: 'Job queued' }) } ``` ## Logging & Monitoring ### Structured Logging ```typescript interface LogContext { userId?: string requestId?: string method?: string path?: string [key: string]: unknown } class Logger { log(level: 'info' | 'warn' | 'error', message: string, context?: LogContext) { const entry = { timestamp: new Date().toISOString(), level, message, ...context } console.log(JSON.stringify(entry)) } info(message: string, context?: LogContext) { this.log('info', message, context) } warn(message: string, context?: LogContext) { this.log('warn', message, context) } error(message: string, error: Error, context?: LogContext) { this.log('error', message, { ...context, error: error.message, stack: error.stack }) } } const logger = new Logger() // Usage export async function GET(request: Request) { const requestId = crypto.randomUUID() logger.info('Fetching markets', { requestId, method: 'GET', path: '/api/markets' }) try { const markets = await fetchMarkets() return NextResponse.json({ success: true, data: markets }) } catch (error) { logger.error('Failed to fetch markets', error as Error, { requestId }) return NextResponse.json({ error: 'Internal error' }, { status: 500 }) } } ``` **Remember**: Backend patterns enable scalable, maintainable server-side applications. Choose patterns that fit your complexity level.

[{"step":"Define your application requirements and constraints","description":"Specify the type of application (e.g., SaaS, e-commerce), expected traffic volume, data relationships, and any compliance requirements (GDPR, HIPAA). Use this to determine initial architectural decisions.","tip":"For prototypes, start with a simpler architecture and scale up as needed. For production systems, involve a DBA early for schema optimization."},{"step":"Select your technology stack components","description":"Choose between REST vs GraphQL, SQL vs NoSQL databases, and caching solutions based on your data access patterns. Use the prompt template to generate architecture recommendations.","tip":"Consider using Next.js API routes if your frontend is Next.js-based for simplified deployment. For microservices, evaluate Express vs Fastify."},{"step":"Implement core patterns","description":"Start with database models, API controllers, and middleware. Use the generated code snippets as a foundation. Implement connection pooling, caching layers, and error handling early.","tip":"Use tools like Prisma for database access if working with multiple database types. Implement health check endpoints (/health) for monitoring."},{"step":"Optimize and test","description":"Profile database queries, test caching effectiveness, and simulate traffic patterns. Use tools like k6 for load testing and pgBadger for PostgreSQL performance analysis.","tip":"Set up continuous profiling with tools like New Relic or Datadog. Implement circuit breakers for external service calls."},{"step":"Deploy and monitor","description":"Containerize your application and deploy to your chosen platform. Set up monitoring for key metrics (response times, error rates, database performance).","tip":"Use infrastructure-as-code tools like Terraform for reproducible environments. Implement canary deployments for new features."}]