AI for Classifying and Routing Requests in Jira and Zendesk

Short answer: AI classification is useful when categories are clearly defined and quality is checked separately for each class, rather than measured by one overall accuracy score. Topic, language, priority, and sentiment are different fields. An emotional email is not necessarily a critical incident, while a calm description may report an outage.

The key is that errors have different costs across classes. Measure missed critical incidents separately, even if they are rare. After reading, you should be able to decide which decisions to automate, prepare categories, check the built-in capabilities of your Jira or Zendesk account, and set up measurement and a gradual rollout.

Which decisions to automate

“Routing” combines several distinct decisions. Separate them because each has its own logic and cost of error.

• Topic. What the request concerns: billing, access, a technical failure, or a return. The topic helps identify the team with the right knowledge.
• Queue or assignee. Where the request goes. Topic influences the queue but does not fully determine it; language, region, customer type, and team workload also matter.
• Language. In multilingual support, language determines who can respond.
• Priority. How urgently the team should respond. It depends on impact on the customer and business, service terms, and service rules.
• Sentiment. How frustrated the customer sounds. This can call for more careful communication, but it is not a priority by itself.

The most common mistake is to confuse sentiment with priority. “Urgent, where is my invoice?!” sounds like an emergency, but it is a routine billing question. “Good morning, no one on our team has been able to log in since this morning” sounds calm but describes an incident affecting the customer's entire company. If priority follows emotion, the first message jumps ahead of the second.

It is therefore useful for a model to identify the topic and the facts that affect priority separately (how many users are affected, whether work has stopped, whether data is at risk). Rules can then assign priority based on those facts and the service terms.

Preparing categories

Classification quality is limited by category quality. If people on the team cannot assign a request to a class consistently, a model will not be able to do so either.

A usable taxonomy needs:

• Non-overlapping definitions. One or two sentences for each category explaining what is included and excluded. For example: “Access: login problems, password resets, user permissions. Excludes a service-wide outage, which is a technical incident.”
• Examples. Several real requests for each category, especially borderline cases.
• An unknown category. A request that does not fit any class should go to a separate manual-review queue, not to the closest match.
• A reasonable number of classes. Use as many as help routing and the team can label reliably. Twenty fine-grained categories that do not change handling only add opportunities for error.

Before connecting AI, run a simple check: have two team members independently label the same sample of requests. Frequent disagreement points to unclear definitions that should be resolved before automation.

Built-in features and a custom classifier

Before building your own solution, check what your support platform already does. Capabilities depend on the product, plan, add-ons, and environment, so verify the description below against your account.

Zendesk. Its intelligent triage feature classifies new requests by topic (previously called “intent” in the interface), sentiment, language, and entities such as product names, and shows classification confidence. According to Zendesk documentation, as of October 2026, classifications are available on Suite and Support Professional plans and above; using them in triggers and other workflows requires the Copilot add-on. Initial classification is based on the ticket subject and first public comment. If dynamic classification is enabled in your account, topic and sentiment can be reevaluated after later end-user replies; language is not reevaluated with each reply. Zendesk also recommends defining a confidence threshold separately for each topic before taking automated action.

Jira Service Management. Atlassian offers AI queue triage: an agent selects several requests, receives suggestions for new request types and field values, reviews them, and applies the chosen changes. A person therefore confirms changes before they take effect. There are limits: a request type is unavailable if AI cannot fill its required fields; the feature requires AI capabilities to be enabled and is unavailable in some environments.

Before deciding, check your account:

• whether classification is available on your plan and whether a separate add-on is needed to use it in automations;
• which fields it fills and whether they map to your categories;
• whether it supports your customers' languages both for classification and for trigger values and reports;
• whether classification uses the first message or the entire conversation;
• whether confidence is visible and usable in rules;
• who can change a classification manually and whether that change is logged for later analysis.

A custom classifier makes sense when the built-in feature is unavailable on your plan, lacks a needed category or language, or must consider data from other systems, such as CRM history or contract terms. Even then, the support platform's fields and routing rules remain the execution mechanism: the model proposes values, and triggers or automations move the request.

Critical errors

Classification errors do not all have the same cost. Consider two groups separately.

A missed critical incident. A request about an outage, data leak, or customer work stoppage goes to an ordinary queue. The response is delayed by hours. This is usually the costliest type of error.

False urgency. An ordinary request gets high priority. It takes attention away from on-call staff and gradually weakens the meaning of priority. The cost is lower, but not zero.

There is a trade-off between these outcomes. Configure a system to miss no critical incidents and it will more often raise the priority of ordinary requests by mistake. The support lead must set this balance with the cost of each error in mind.

Practical safeguards:

• Check key incident signals (widespread unavailability, security, data loss) with a separate rule, not just the general classifier.
• Send low-confidence classifications to manual review.
• Run an additional priority check for customers with special service terms.
• Review every late-detected critical request individually: what the text said, how the model classified it, and which rule should have caught it. Add each case to the test set immediately.

Automatic routing should not prevent agents from raising priority quickly by hand. Someone who sees an outage should be able to change priority in one action, and that decision should override the model.

Measurement

Overall accuracy—the share of requests classified correctly—is convenient but risky: it can hide the most expensive errors. The fictional training example below shows how.

Training example: how overall accuracy hides missed incidents

Labeled sample: 200 requests. There are 196 ordinary requests (billing, access, technical failure, return) and 4 critical incidents.

The classifier correctly identifies 188 of the 196 ordinary requests and only 1 of the 4 critical incidents.

• Overall accuracy: (188 + 1) / 200 = 189 / 200 = 94.5%.
• Recall for “critical incident”: 1 / 4 = 25%. In other words, three of four incidents went to the wrong place.

The 94.5% figure looks good, but the result is unacceptable for the business. That is why metrics must be calculated for each class.

Where should you get a labeled sample? The simplest source is closed requests where agents already chose a queue and priority. Review them, because historical decisions can also be wrong. Rare critical classes hardly appear in a random sample, so add them separately: gather all known incidents from a longer period to get enough examples for each critical class. Ideally, label the data before reviewers see the model's prediction, or they may unconsciously agree with it.

Training table for reviewing individual requests

Metrics to track:

• Precision by class: of the requests the model assigned to a class, what share actually belong there?
• Recall by class: what share of all requests in a class did the model find? This is the main metric for critical classes.
• Confusion matrix: which classes the model confuses with one another, helping identify definitions to clarify.
• Manual reassignment rate: how often agents move a request to another queue after automatic routing.
• Time to first response and resolution by queue before and after launch, accounting for seasonality.

Rollout and updates

Enable classification gradually so you can catch errors before they affect customers.

1. Shadow mode. The model classifies requests, but the result is only written to a field and does not affect routing. The team works as before while you compare predictions with agents' actual decisions.
2. Limited queue. Turn on automatic routing for one or two categories with the best metrics and low error cost. Keep critical classes under manual control or separate rules at this stage.
3. Expansion. Add categories only after their precision and recall on a fresh labeled sample reach the threshold agreed with the support lead.
4. Regular review. Requests change as the product, problems, and seasons change. Recalculate metrics regularly and update the taxonomy and examples when manual reassignment or “unknown” cases increase.

Conclusion

AI classification reduces manual sorting only when categories are clear, topic is separate from priority and sentiment, and quality is measured by class. A missed critical incident needs its own metric and safeguards regardless of overall accuracy.

What you can do in your process:

1. Separate topic, queue, language, priority, and sentiment. Decide which are model decisions and which are rule-based.
2. Define non-overlapping categories with examples and an “unknown” category. Check that two people label cases consistently.
3. Verify which classification and routing features are available on your specific Jira or Zendesk plan.
4. Build a labeled sample with a separate group of critical requests, and track precision and recall by class, not only overall accuracy.
5. Start in shadow mode, then enable a limited queue with a low cost of error.
6. Assign an owner to review metrics, taxonomy, and priority rules regularly.

If you want to reduce manual request sorting, the BotLabs team can review your queue in a 15-minute consultation. We can discuss categories, priority rules, critical requests that must not be missed, and which capabilities already exist in your support platform. The next step is to choose a queue and sample for a shadow-mode pilot.

Frequently asked questions