But once you start building real-world LLM applications, you'll realize the AI model is only one small part of the system.

Recently, I came across these six questions that senior engineers often discuss:

• What happens when the LLM call times out?

• How do you stop users from burning hundreds of dollars in tokens?

• What if your vector database retrieves the wrong information?

• How do you cache an AI that gives different answers every time?

• What do you log when someone says, "The answer was wrong"?

• Your API returned 200 OK, but the AI still gave a useless answer. Now what?

At first, these questions sounded complicated.

After understanding them, I realized they are actually software engineering problems, not AI problems.

Let's understand each one in the simplest way possible.

1. What happens when the LLM call times out?

Imagine your backend sends a request to GPT.

User
   │
   ▼
Backend
   │
   ▼
OpenAI API

Normally, the model responds within a few seconds.

But sometimes the model takes too long.

After waiting for 30 or 60 seconds, your backend stops waiting and returns an error.

This is called a timeout.

The important thing to understand is:

Your backend stopped waiting, but the LLM might still be generating a response.

Now imagine the user clicks Generate again.

Instead of one expensive request, you now have two requests running at the same time.

That means higher cost and unnecessary work.

How companies handle it

Instead of making the user wait forever, they usually:

• Run the LLM request in the background.

• Give the user a job ID.

• Let the frontend check whether the result is ready.

• Retry only when it's safe.

This makes the application much more reliable.

2. How do you stop one user from spending $400 in tokens?

Every API call costs money.

Now imagine someone writes a script like this:

while True:
    ask_ai()

Thousands of requests later, your OpenAI bill becomes much larger than expected.

One careless or malicious user can consume a huge portion of your budget.

How companies handle it

They don't allow unlimited usage.

Common techniques include:

• Rate limiting (for example, 20 requests per minute)

• Daily request limits

• Monthly token limits

• Maximum context size

• Budget alerts for unusual spending

These limits protect both the application and the company.

3. Where does the Vector Database live, and what if it retrieves the wrong information?

Many LLM applications use RAG (Retrieval-Augmented Generation).

The flow usually looks like this:

User Question
      │
      ▼
Vector Database
      │
Relevant Documents
      │
      ▼
LLM
      │
      ▼
Final Answer

The Vector Database stores document embeddings and helps find the most relevant information.

But sometimes it retrieves the wrong documents.

Imagine asking:

"What is our vacation policy?"

Instead of retrieving the vacation policy, it retrieves the parking policy.

The LLM doesn't know it's the wrong document.

It simply answers confidently based on what it received.

This is why people often say:

Garbage in → Garbage out.

How companies handle it

They improve retrieval by:

• Splitting documents into better chunks

• Using better embedding models

• Combining keyword search with vector search

• Re-ranking retrieved documents

• Refusing to answer when confidence is too low

The goal isn't to make the LLM smarter.

The goal is to provide it with better information.

4. How do you cache something that gives different answers every time?

Normally, caching is simple.

If someone asks:

What is 2 + 2?

The answer is always 4.

So we can save the answer and reuse it later.

But LLMs don't always behave like that.

Ask:

"Write me a motivational quote."

Today, you may get one answer.

Tomorrow, you may get a completely different one.

Both are correct.

That's because LLMs are non-deterministic.

So what do companies cache?

Instead of caching everything, they cache only things that are expected to stay the same.

For example:

• Document embeddings

• Vector search results

• Language translations

• Text classification

• PDF summaries

They usually avoid caching creative tasks like stories, poems, or jokes because users expect fresh responses.

The idea is simple:

Cache predictable work. Don't cache creativity.

5. What do you log when someone says, "The answer was wrong"?

In traditional software, debugging is usually straightforward.

If the application crashes, you look at the error logs.

But AI applications are different.

The system may work perfectly, yet still produce a bad answer.

Now imagine a user says:

"The chatbot gave me the wrong refund policy."

How do you investigate?

Companies log the entire AI journey.

Instead of logging only errors, they also save things like:

• The user's question

• The system prompt

• Retrieved documents

• Model name

• Temperature

• Token usage

• Response time

• Final AI response

This allows developers to replay the request and understand what happened.

Maybe the wrong document was retrieved.

Maybe the prompt had an issue.

Maybe the model hallucinated.

Without logs, finding the real cause becomes almost impossible.

6. Your monitoring says 200 OK, but users still get nonsense.

This is one of the biggest differences between traditional software and AI applications.

Imagine your backend returns:

HTTP 200 OK

Normally, that means everything worked.

But in AI systems, 200 OK only means the request was processed successfully.

It does not guarantee the answer is correct.

For example:

User asks:

"What is the capital of Pakistan?"

AI replies:

"London."

Technically:

• API worked

• Server didn't crash

• Model responded

Everything looks healthy.

But the answer is obviously wrong.

How companies handle it

They monitor more than just server health.

They also look at AI quality.

Some common approaches include:

• 👍 / 👎 feedback buttons

• Checking whether answers match the retrieved documents

• Using another LLM to review responses

• Monitoring how users interact after receiving an answer

Good AI monitoring measures whether the answer helped the user, not just whether the server responded successfully.

Final Thoughts

Building an LLM application is much more than calling an API.

Most production challenges are actually software engineering problems:

• Handling failures

• Controlling costs

• Retrieving the right information

• Caching intelligently

• Logging useful data

• Measuring answer quality

Understanding these concepts won't just help you build better AI applications it will also help you in backend and system design interviews.

The AI model is important.

But the engineering around it is what makes an AI application reliable.

In today's digital landscape, website and server uptime is critical. Downtime can lead to lost revenue, poor user experience, and damaged reputation. To address this, I built Service Monitoring Pro—a Django-based web application that continuously monitors websites and servers,notifies users (using emails) when downtime occurs, and provides an alert dashboard.

In this article, I’ll walk you through how I built this app using Django, Celery, Redis, and Docker.🚀

Monitoring Mechanism

The system is designed to monitor websites and servers using asynchronous background tasks. It works as follows:

1. Website Monitoring: The system periodically sends HTTP requests to configured websites and checks for responses.

2. Server Monitoring: It utilizes ping commands to determine server availability.

3. Failure Detection: If a website or server fails to respond within a set threshold, an alert is triggered.

4. Alert System: Email notifications are sent to the user when a service goes down.

5. Real-time Updates: Using WebSockets, the dashboard reflects live status updates of monitored services.

Notification System

Timely alerts are essential in server and website monitoring. Our system ensures that users receive instant notifications through:

• Email Alerts: Whenever a service becomes unresponsive, an email is sent with details of the failure.

• Dashboard Alerts: A real-time status update appears on the dashboard to keep users informed at all times.

Implementation Details

Models for Storing Website Data

Create a model to store the websites and servers that we want to monitor. Each entry will store the URL or server address, the status (whether it's up or down), and the frequency at which we want to check it. Here is the model code:

from django.db import models
from django.contrib.auth.models import User
from django.utils.timezone import now

class Website(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    url = models.URLField(unique=True)
    status = models.CharField(max_length=10, default="UNKNOWN")
    interval = models.IntegerField(choices=INTERVAL_CHOICES, default=60) 
    last_checked = models.DateTimeField(auto_now=True)
    monitoring_type=models.CharField(max_length=50,default="server")

    def __str__(self):
        return self.url

Explanation:

• url: This field stores the URL or address of the website or server.

• status: This field holds the current status of the website (e.g., "UP", "DOWN").

• interval: Defines how frequently the system should check the website (e.g., every 1 minute, every 5 minutes).

• last_checked: The timestamp of the last time the website was checked.

• monitoring_type: Whether we are monitoring a website or server.

Celery for Asynchronous Task Execution

To check the website’s or server’s status at regular intervals, we will use Celery, a powerful asynchronous task queue. Celery helps run periodic tasks in the background, ensuring that the system can check the website without blocking the user’s workflow.

Creating a file celery.py in your Django project directory:

import os
from celery import Celery
from celery.schedules import crontab
from datetime import timedelta

# Set Django settings for Celery
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "process_monitoring.settings")

app = Celery("process_monitoring")

# Load configuration from Django settings
app.config_from_object("django.conf:settings", namespace="CELERY")

# Make sure Celery is using Redis
app.conf.broker_url = os.getenv("CELERY_BROKER_URL", "redis://redis:6379/0")

# Discover tasks in installed Django apps
app.autodiscover_tasks()

# Define periodic tasks using Celery Beat
app.conf.beat_schedule = {
    'check-websites-every-30-sec': {
        'task': 'monitoring.tasks.check_30_sec',
        'schedule': timedelta(seconds=30),
    },
    'check-websites-every-1-min': {
        'task': 'monitoring.tasks.check_1_min',
        'schedule': timedelta(minutes=1),
    },
    'check-websites-every-2-min': {
        'task': 'monitoring.tasks.check_2_min',
        'schedule': timedelta(minutes=2),
    },
    'check-websites-every-5-min': {
        'task': 'monitoring.tasks.check_5_min',
        'schedule': timedelta(minutes=5),
    },
}

This code sets up Celery for managing periodic tasks in a Django project, specifically for monitoring website statuses at different intervals. It configures Celery with Redis as the message broker, enabling asynchronous task execution in the background without affecting the main application’s performance. The periodic tasks are defined using Celery Beat, which allows tasks like check_30_sec, check_1_min, check_2_min, and check_5_min to run at intervals of 30 seconds, 1 minute, 2 minutes, and 5 minutes, respectively. This configuration ensures that website monitoring occurs automatically at the specified intervals, providing real-time status updates to users.

Create a periodic task to check the website’s status at the interval defined in the Website model. In tasks.py

import requests
import socket
import subprocess
import logging
from django.utils.timezone import now
from django.core.mail import send_mail
from django.conf import settings
from celery import shared_task
from .models import Website
from alert.models import Alert
import monitoring.utils as utils

logger = logging.getLogger(__name__)



def check_websites(interval):
    websites = Website.objects.filter(interval=interval)
    logger.info(f"Checking {websites.count()} websites for {interval}-second interval")

    for website in websites:
        monitoring_type=website.monitoring_type

        if monitoring_type=="website":
            website_status=utils.check_website_status(website.url)
            if website.status!=website_status:
                utils.send_alert(website,website_status)
            website.status=website_status
            website.last_checked=now()
            website.save()
            continue
        elif monitoring_type=="server":
            hostname = website.url.replace("http://", "").replace("https://", "").split("/")[0]
            new_status=utils.check_server_status(hostname,website.url)
            if website.status != new_status:
                utils.send_alert(website, new_status)
            website.status = new_status
            website.last_checked = now()
            website.save()
            continue
        elif monitoring_type=="both":
            hostname = website.url.replace("http://", "").replace("https://", "").split("/")[0]
            new_status_server=utils.check_server_status(hostname,website.url)
            new_status_website=utils.check_website_status(website.url)
            new_status=""
            if new_status_server=="UP" and new_status_website=="UP":
                new_status="UP"
            else:
                new_status="DOWN"
            if website.status!=new_status:
                utils.send_alert(website,new_status)
            website.status=new_status
            website.last_checked=now()
            website.save()
    return f"Checked {websites.count()} websites for {interval}-second interval"

# 6️⃣ CELERY TASKS (Scheduled Checks)
@shared_task
def check_30_sec():
    return check_websites(30)

@shared_task
def check_1_min():
    return check_websites(60)

@shared_task
def check_2_min():
    return check_websites(120)

@shared_task
def check_5_min():
    return check_websites(300)

This tasks.py file defines periodic Celery tasks for checking the status of websites and servers at different intervals. The check_websites function queries the Website model to fetch websites based on their monitoring interval. It checks the status of each website or server (depending on the monitoring_type) by calling helper functions from utils. If there is a change in status (e.g., from UP to DOWN), it triggers an alert using the send_alert function. The four Celery tasks (check_30_sec, check_1_min, check_2_min, and check_5_min) are scheduled to run at 30-second, 1-minute, 2-minute, and 5-minute intervals, respectively, ensuring continuous monitoring.

The utils.py file provides functions for monitoring website and server health:

import requests
import socket
import subprocess
import logging
from django.utils.timezone import now
from django.core.mail import send_mail
from django.conf import settings
from alert.models import Alert
logger = logging.getLogger(__name__)


class ServerChecking:
    def is_server_reachable(self,hostname):
        try:
            output = subprocess.run(["ping", "-n", "2", hostname], capture_output=True, text=True, timeout=5)
            return "Reply from" in output.stdout
        except Exception as e:
            return False


    # 2️⃣ PORT CHECK FUNCTION (Checks if HTTP/HTTPS are Running)
    def is_port_open(self,hostname, port):
        try:
            with socket.create_connection((hostname, port), timeout=5):
                return True
        except (socket.timeout, socket.error):
            return False

    # 3️⃣ WEBSITE STATUS CHECK FUNCTION
    def check_website_status(self,url):
        try:
            response = requests.get(url, timeout=5)
            return response.status_code not in [502, 503, 504]
        except requests.RequestException:
            return False

def check_website_status(url):
    status=""
    try:
        response = requests.get(url, timeout=5)  # Set timeout to avoid long delays
        if response.status_code>=200 and response.status_code<400:
            status="UP"
            return status
        else:
            status="DOWN"
            return status
    except requests.exceptions.ConnectionError:
        status="DOWN"
        return status
    except requests.exceptions.Timeout:
        status="DOWN"
        return status
    except requests.exceptions.RequestException:
        status="DOWN"
        return status


def check_server_status(hostname,url):
    check=ServerChecking()
    server_up = check.is_server_reachable(hostname)
    port_80_open = check.is_port_open(hostname, 80)
    port_443_open = check.is_port_open(hostname, 443)
    website_up = check.check_website_status(url)
    new_status = "UP" if server_up and website_up and (port_80_open or port_443_open) else "DOWN"
    return new_status

def send_alert(website, status):
    message = f"Your server hosting {website.url} is currently {status}."
    Alert.objects.create(user=website.user, url=website.url, status=status, message=message, is_read=False)

    if status == "DOWN":
        try:
            send_alert_email(website.url,website.user.email)
            logger.info(f"Email alert sent for {website.url}")
        except Exception as e:
            logger.error(f"Failed to send alert email for {website.url}: {e}")

def send_alert_email(website_url,email):
    subject = "⚠️ Website Server Down Alert"
    message = f"The website Server {website_url} is currently DOWN. Please check it immediately!"
    from_email = settings.DEFAULT_FROM_EMAIL
    recipient_list = [email]  # Change this to the user's email

    try:
        send_mail(subject, message, from_email, recipient_list)
        logger.info(f"Alert email sent for {website_url}")
    except Exception as e:
        logger.error(f"Failed to send email for {website_url}: {e}")

1. ServerChecking Class:

   • is_server_reachable: Pings the server to check if it's reachable.

   • is_port_open: Checks if HTTP (port 80) or HTTPS (port 443) is open.

   • check_website_status: Checks if a website is accessible via a GET request.

2. check_website_status: A function that checks website accessibility and returns "UP" or "DOWN" based on the status code or errors.

3. check_server_status: Combines the checks from ServerChecking to assess the server and website status, returning "UP" or "DOWN".

4. send_alert: Creates an alert and sends an email notification if the website's status is "DOWN".

5. send_alert_email: Sends an email alert to the user about the website being down.

In essence, the file monitors the health of websites/servers and sends notifications if issues are detected.

Want to check out the code? Here’s the GitHub Repo.