Merchant ticket triage (BG/EN)

The model was not consulted: the stop rules ended the loop.

claude-sonnet-5-5: Macro F1 0.825 is below the 0.92 target. The weak classes are complaints (F1 0.55) and other (F1 0.69). Complaints are mislabeled as returns (damaged goods with demands for an escalation or plan) and as other (legal, GDPR, security, contract threats). Other is over-predicted: legit product_questions (wholesale, B2B, legal-sounding) and ordinary topics (shipping, payments) with legal or formal vocabulary fall into it. Payments are confused with complaints and returns when the tone is angry or the subject is a refund, a fee or a subscription. Legal vocabulary in orders and shipping tickets pushes predictions to the wrong category. Tone and keywords are overriding the main topic. (model's own stop opinion: continue, not used)

• complaints_vs_returns x200 Bg and en tickets about damaged or broken goods on arrival, where the customer's main demand is accountability, an escalation, a formal response, or a compensation plan rather than a standard return or exchange. Contrast them with plain return or exchange requests that stay in returns. Vary the tone, the urgency and the threats to leave.
• complaints_vs_other_legal x200 Tickets with legal, GDPR, data-privacy, security-breach, contract or arbitration threats about the shop's conduct or service, labeled complaints with high urgency and needs_human true. Contrast them with true 'other' tickets: neutral non-shop topics, minor site or app bugs, cosmetic feedback, partnership or spam messages. Use both bg and en, and vary the formality.
• payments_vs_complaints_returns x180 Angry or neutral tickets whose core issue is money: double charges, unexpected fees, subscription billing, refunds not received after a return, chargeback threats. Label them payments even when the tone is hostile or the ticket mentions returns, shipping or site problems. Include some calm fee-explanation questions with needs_human true. Contrast with complaints about service quality that mention no money problem.
• product_questions_vs_other x170 Pre-purchase questions about wholesale or bulk pricing, corporate gift cards, quotes, specs, certifications, and regulatory or warranty validity of products. Label them product_questions, with low urgency, and needs_human true only when legal or regulatory advice is requested. Contrast with truly off-topic 'other' tickets. Use bg and en, formal and casual.
• legal_words_orders_shipping x150 Orders and shipping tickets that contain legal, corporate or compliance vocabulary, such as a lawyer requesting a delivery confirmation, a company HQ address, freight liability or an invoice discrepancy. The label follows the main topic (address change on an unshipped order -> orders; delivery date or transit liability -> shipping). Set needs_human true only when a person must decide or the matter is legal.
• other_minor_bugs_boundary x100 Low-urgency feedback about site or app glitches, cosmetic issues, checkout freezes that resolved themselves, and general suggestions, labeled other with needs_human false. Contrast them with real complaints about orders or service, which carry real harm and a demand for resolution. Mix bg and en.

claude-sonnet-5-5: Macro F1 0.844 is up from 0.825 but still below the 0.92 target. Complaints is the weakest class (recall 0.5, F1 0.67): half of the tickets fall into other (legal, GDPR, data breach, contract threats, and a persistent site-crash frustration case) or into returns, shipping and payments when the damage or demand is strong. Other is over-predicted (precision 0.67), absorbing complaints and product_questions (wholesale and B2B). Shipping, orders and returns are confused when the ticket involves damaged goods in transit, late delivery combined with a return demand, address changes, or legal vocabulary. Payments is confused with shipping and returns when the issue is a fee or a charge. Tone and surface keywords still override the main topic, so more contrastive data on these boundaries is needed. The urgency (0.84) and needs_human fields also have errors, for example urgency inflated by 'legal' words and needs_human missed on fee disputes. (model's own stop opinion: continue, not used)

• complaints_vs_other x220 Bg and en tickets where the customer is clearly unhappy with the shop's service, conduct or platform and wants a person to respond: persistent site or checkout failures causing real frustration, data-protection or GDPR disputes, security concerns, contract or terms disputes, and threats of formal action. Label complaints with needs_human true and urgency matched to deadlines or threats (high when a deadline, money loss or threat to leave is present, otherwise normal). Contrast with genuine other tickets: spam, partnership offers, neutral non-shop topics, one-off minor glitches or suggestions that need no resolution, labeled other with needs_human false. Vary formality and length.
• complaints_vs_returns_shipping x200 Tickets about goods that arrived broken or late where the customer's main demand is accountability, escalation, compensation or a threat to leave, labeled complaints. Contrast with tickets in which the main ask is a concrete logistics action: a return or exchange labeled returns, a delivery status or courier problem labeled shipping. The label follows what the customer wants solved first. Include cases that combine damage, courier blame and a request for a supervisor or a refund. Use both languages and vary the tone.
• shipping_vs_orders x170 Contrast tickets about changing or correcting an order before dispatch (address fix, item change, cancellation, an order not visible or confirmed in the system) labeled orders, with tickets about transit, delivery dates, lost or delayed parcels, delivery confirmations and courier issues labeled shipping. Include cases with legal, corporate or compliance wording, lawyers or company HQ addresses, where the label still follows the main topic. Include some where payment was made but the order is not shown as dispatched. Set urgency by real deadlines or money loss, not by formal vocabulary. Use bg and en.
• payments_fees_vs_shipping_returns x170 Tickets whose core issue is money: unexpected delivery fees or charges on a bill, repeated charges after a return, refunds not received, double charges, billing explanations. Label payments even when delivery, shipping or return words appear. Set needs_human true when a person must decide on a refund, fee reversal or fee explanation. Contrast with shipping tickets about delivery status and returns tickets about how to send goods back. Include angry and calm tones, with threats to involve authorities for the high-urgency cases. Mix bg and en.
• product_questions_b2b_vs_other x150 Pre-purchase questions about wholesale or bulk prices, price lists, corporate gift cards, quotes by email, minimum order quantities, product specs and certifications. Label product_questions with low urgency and needs_human false (true only if legal or regulatory advice is requested). Contrast with truly off-topic or spam other tickets. Use bg and en, casual and formal, with business context such as cafes, shops and companies.
• urgency_needs_human_calibration x90 Tickets across all categories that test the urgency and needs_human rules. Formal, legal or corporate wording without a deadline or money loss stays at normal urgency. High urgency requires lost money, a missed deadline or a threat to leave. Plain questions are low. needs_human is true for refunds, fee disputes, complaints and legal matters, and false for routine status checks, address fixes and information questions. Use bg and en with a balanced mix of labels.

The model was not consulted: the stop rules ended the loop.