Nirnoy নির্ণয়

Examples · real model output

What you can ask Nirnoy

Nirnoy answers three kinds of question about a message, a record or a conversation, in Bangla, Banglish or English. Ask one question or many in the same request. Every answer below is the model's real output for the request shown (Nirnoy-flash on an L4, Nirnoy 12B on an L40S); we wrote the messages for this page.

Yes / no"type": "noul"

The probability that the answer is yes. Optionally say what yes and no mean.

e.g. Does the customer want a refund?

Multiple choice"type": "choice"

One of 2 to 255 options you name, with a probability for each. Options can carry a short description.

e.g. Which team should take this ticket?

Score"type": "score"

A level on an ordered scale of 2 to 10 levels you describe, with a probability for each level.

e.g. How urgent is this, from 1 to 5?

Yes / no

Ask a yes-or-no question

A noul question returns the probability of yes. Here: does the customer want their money back?

Message

ভাই অর্ডারটা বাতিল করেন, প্রোডাক্ট একদম বাজে এসেছে। আমার টাকাটা ফেরত দেন।

Does the customer ask for their money back? · yes / no

Nirnoy-flash
yes
99%
no
<1%
Nirnoy 12B
yes
99%
no
<1%

Nirnoy-flash: 47 ms · Nirnoy 12B: 45 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "ভাই অর্ডারটা বাতিল করেন, প্রোডাক্ট একদম বাজে এসেছে। আমার টাকাটা ফেরত দেন।",
  "questions": {
    "refund": {
      "type": "noul",
      "instructions": "Does the customer ask for their money back?"
    }
  }
}

Yes / no

Define what yes and no mean

Give your own meaning for yes and no, in any language. Here, in Bangla: does this chat need a human agent?

Message

Apnader bot amar kotha bujhe na. Ekjon manush er sathe kotha bolte chai, please.

এই বার্তার জন্য কি একজন মানুষ এজেন্ট দরকার? · yes / no

Nirnoy-flash
yes
99%
no
<1%
Nirnoy 12B
yes
99%
no
<1%

Nirnoy-flash: 53 ms · Nirnoy 12B: 49 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "Apnader bot amar kotha bujhe na. Ekjon manush er sathe kotha bolte chai, please.",
  "questions": {
    "needs_human": {
      "type": "noul",
      "instructions": "এই বার্তার জন্য কি একজন মানুষ এজেন্ট দরকার?",
      "criteria": {
        "true": "বট সমাধান করতে পারবে না বা গ্রাহক মানুষ চাইছেন",
        "false": "স্বয়ংক্রিয় উত্তরে চলবে"
      }
    }
  }
}

Multiple choice

Pick one of your options

A choice question returns one option and a probability for every option. The options are just a list.

Message

Delivery khub fast chilo, kintu packaging ta bhanga chilo.

What is the overall sentiment of this review? · choice · 4 options

Nirnoy-flash
positive
1%
negative
1%
neutral
1%
mixed
97%
Nirnoy 12B
positive
1%
negative
2%
neutral
1%
mixed
96%

Nirnoy-flash: 53 ms · Nirnoy 12B: 49 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "Delivery khub fast chilo, kintu packaging ta bhanga chilo.",
  "questions": {
    "sentiment": {
      "type": "choice",
      "instructions": "What is the overall sentiment of this review?",
      "criteria": [
        "positive",
        "negative",
        "neutral",
        "mixed"
      ]
    }
  }
}

Multiple choice

Describe each option

Add a short description to each option and Nirnoy follows your definitions: here, which team should take the ticket.

Message

আমার বিকাশ থেকে ৫০০ টাকা কেটে গেছে কিন্তু অর্ডার কনফার্ম হয়নি।

কোন টিম এই টিকেটটি দেখবে? · choice · 4 options

Nirnoy-flash
billing
98%
delivery
1%
technical
1%
sales
1%
Nirnoy 12B
billing
98%
delivery
1%
technical
1%
sales
1%

Nirnoy-flash: 53 ms · Nirnoy 12B: 49 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "আমার বিকাশ থেকে ৫০০ টাকা কেটে গেছে কিন্তু অর্ডার কনফার্ম হয়নি।",
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "কোন টিম এই টিকেটটি দেখবে?",
      "criteria": {
        "billing": "পেমেন্ট, টাকা কাটা, রিফান্ড",
        "delivery": "ডেলিভারি, কুরিয়ার, ঠিকানা",
        "technical": "অ্যাপ, লগইন, ওটিপি বা অ্যাকাউন্টের সমস্যা",
        "sales": "দাম, স্টক, অফার, নতুন অর্ডার"
      }
    }
  }
}

Multiple choice

Many options at once

A choice question can have up to 255 options. Here, all 64 districts of Bangladesh in one question.

Message

Sylhet er Zindabazar e delivery dite koto din lagbe?

Which district is the customer in? · choice · 64 options

Nirnoy-flash
Sylhet
75%
Manikganj
9%
Chattogram
2%
Sirajganj
1%

and 60 more options, each below 1%

Nirnoy 12B
Sylhet
89%
Manikganj
2%
Sirajganj
1%
Tangail
<1%

and 60 more options, each below 1%

Nirnoy-flash: 108 ms · Nirnoy 12B: 84 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "Sylhet er Zindabazar e delivery dite koto din lagbe?",
  "questions": {
    "district": {
      "type": "choice",
      "instructions": "Which district is the customer in?",
      "criteria": [
        "Bagerhat",
        "Bandarban",
        "Barguna",
        "Barishal",
        "Bhola",
        "Bogura",
        "Brahmanbaria",
        "Chandpur",
        "Chapai Nawabganj",
        "Chattogram",
        "Chuadanga",
        "Cox's Bazar",
        "Cumilla",
        "Dhaka",
        "Dinajpur",
        "Faridpur",
        "Feni",
        "Gaibandha",
        "Gazipur",
        "Gopalganj",
        "Habiganj",
        "Jamalpur",
        "Jashore",
        "Jhalokati",
        "Jhenaidah",
        "Joypurhat",
        "Khagrachhari",
        "Khulna",
        "Kishoreganj",
        "Kurigram",
        "Kushtia",
        "Lakshmipur",
        "Lalmonirhat",
        "Madaripur",
        "Magura",
        "Manikganj",
        "Meherpur",
        "Moulvibazar",
        "Munshiganj",
        "Mymensingh",
        "Naogaon",
        "Narail",
        "Narayanganj",
        "Narsingdi",
        "Natore",
        "Netrokona",
        "Nilphamari",
        "Noakhali",
        "Pabna",
        "Panchagarh",
        "Patuakhali",
        "Pirojpur",
        "Rajbari",
        "Rajshahi",
        "Rangamati",
        "Rangpur",
        "Satkhira",
        "Shariatpur",
        "Sherpur",
        "Sirajganj",
        "Sunamganj",
        "Sylhet",
        "Tangail",
        "Thakurgaon"
      ]
    }
  }
}

Score

Score on your own scale

A score question uses ordered levels you describe (2 to 10). Nirnoy returns a probability for each level.

Message

আমার বিকাশ অ্যাকাউন্ট থেকে কেউ টাকা তুলে নিয়েছে, আমি কোনো ওটিপি দেইনি! এখনই অ্যাকাউন্ট বন্ধ করুন।

এই কেসটি কতটা জরুরি? · score · 5 levels

Nirnoy-flash
1 · একদম জরুরি নয়
<1%
2 · কম জরুরি
<1%
3 · মাঝারি
<1%
4 · বেশ জরুরি
<1%
5 · এখনই ব্যবস্থা দরকার (ক্ষতি, টাকা হারানো বা সেবা বন্ধ)
99%
Nirnoy 12B
1 · একদম জরুরি নয়
<1%
2 · কম জরুরি
<1%
3 · মাঝারি
<1%
4 · বেশ জরুরি
<1%
5 · এখনই ব্যবস্থা দরকার (ক্ষতি, টাকা হারানো বা সেবা বন্ধ)
99%

Nirnoy-flash: 54 ms · Nirnoy 12B: 49 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "আমার বিকাশ অ্যাকাউন্ট থেকে কেউ টাকা তুলে নিয়েছে, আমি কোনো ওটিপি দেইনি! এখনই অ্যাকাউন্ট বন্ধ করুন।",
  "questions": {
    "urgency": {
      "type": "score",
      "instructions": "এই কেসটি কতটা জরুরি?",
      "criteria": [
        "একদম জরুরি নয়",
        "কম জরুরি",
        "মাঝারি",
        "বেশ জরুরি",
        "এখনই ব্যবস্থা দরকার (ক্ষতি, টাকা হারানো বা সেবা বন্ধ)"
      ]
    }
  }
}

Score

Turn a review into stars

The same idea for a star rating: five described levels, one answer per level.

Message

Kapor er quality thik ache, kintu size ektu choto. Dam hisebe cholbe.

How many stars would this customer give? · score · 5 levels

Nirnoy-flash
1 · very bad
<1%
2 · bad
2%
3 · okay
66%
4 · good
29%
5 · excellent
2%
Nirnoy 12B
1 · very bad
<1%
2 · bad
1%
3 · okay
50%
4 · good
48%
5 · excellent
<1%

Nirnoy-flash: 53 ms · Nirnoy 12B: 49 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "Kapor er quality thik ache, kintu size ektu choto. Dam hisebe cholbe.",
  "questions": {
    "stars": {
      "type": "score",
      "instructions": "How many stars would this customer give?",
      "criteria": [
        "very bad",
        "bad",
        "okay",
        "good",
        "excellent"
      ]
    }
  }
}

Many questions

Ask several questions in one request

Send all your questions with one message. The answers come back together, each question answered on its own.

Message

Order #4521 ekhono ase nai, 7 din hoye gelo. Ar koto wait korbo? Taka ferot chai.

What does the customer mainly want? · choice · 6 options

Nirnoy-flash
return_refund_cancel
97%
delivery
2%
complaint
1%
payment
<1%

and 2 more options, each below 1%

Nirnoy 12B
return_refund_cancel
90%
delivery
5%
complaint
2%
product_inquiry
1%

and 2 more options, each below 1%

What is the overall sentiment of this message? · choice · 4 options

Nirnoy-flash
positive
1%
negative
94%
neutral
2%
mixed
3%
Nirnoy 12B
positive
1%
negative
94%
neutral
2%
mixed
3%

How urgent is this? · score · 5 levels

Nirnoy-flash
1 · not urgent
<1%
2 · low
1%
3 · medium
4%
4 · high
42%
5 · act now
53%
Nirnoy 12B
1 · not urgent
<1%
2 · low
1%
3 · medium
6%
4 · high
66%
5 · act now
27%

Does the customer ask for their money back? · yes / no

Nirnoy-flash
yes
99%
no
<1%
Nirnoy 12B
yes
99%
no
<1%

Should a human agent take over this chat? · yes / no

Nirnoy-flash
yes
99%
no
1%
Nirnoy 12B
yes
98%
no
2%

Nirnoy-flash: 422 ms · Nirnoy 12B: 297 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "Order #4521 ekhono ase nai, 7 din hoye gelo. Ar koto wait korbo? Taka ferot chai.",
  "questions": {
    "intent": {
      "type": "choice",
      "instructions": "What does the customer mainly want?",
      "criteria": {
        "product_inquiry": "a question about a product: price, stock, size, offers",
        "delivery": "where the order is, when it arrives, delivery charge",
        "payment": "payment problems, bKash, Nagad, cash on delivery",
        "return_refund_cancel": "cancel, return, exchange or get money back",
        "complaint": "a complaint about the product or service",
        "other": "anything else"
      }
    },
    "sentiment": {
      "type": "choice",
      "instructions": "What is the overall sentiment of this message?",
      "criteria": [
        "positive",
        "negative",
        "neutral",
        "mixed"
      ]
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this?",
      "criteria": [
        "not urgent",
        "low",
        "medium",
        "high",
        "act now"
      ]
    },
    "refund": {
      "type": "noul",
      "instructions": "Does the customer ask for their money back?"
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Should a human agent take over this chat?"
    }
  }
}

Structured input

Send a record, not just text

The input can be JSON: an order, a policy and the customer's message together, and questions can point to fields. On the next step Nirnoy-flash is unsure (50%) and Nirnoy 12B is surer (88%): with a threshold, only the unsure answer would go to a person.

Record (JSON)
{
  "order": {
    "id": "A-1093",
    "item": "shoes",
    "status": "delivered",
    "delivered": "3 days ago",
    "price_tk": 1450,
    "payment": "bKash"
  },
  "policy": "পণ্য হাতে পাওয়ার ৭ দিনের মধ্যে ফেরত দেওয়া যাবে। ব্যবহার করা পণ্য ফেরত নেওয়া হয় না।",
  "message": "জুতাটা একবারও পরিনি, সাইজ বড় হয়েছে। ফেরত দিতে চাই।"
}

Is this return allowed under `policy`? · yes / no

Nirnoy-flash
yes
82%
no
18%
Nirnoy 12B
yes
98%
no
2%

What should the agent do next? · choice · 4 options

Nirnoy-flash
approve_return
50%
ask_for_photos
18%
reject
17%
escalate
15%
Nirnoy 12B
approve_return
88%
ask_for_photos
7%
reject
4%
escalate
1%

Nirnoy-flash: 107 ms · Nirnoy 12B: 82 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": {
    "order": {
      "id": "A-1093",
      "item": "shoes",
      "status": "delivered",
      "delivered": "3 days ago",
      "price_tk": 1450,
      "payment": "bKash"
    },
    "policy": "পণ্য হাতে পাওয়ার ৭ দিনের মধ্যে ফেরত দেওয়া যাবে। ব্যবহার করা পণ্য ফেরত নেওয়া হয় না।",
    "message": "জুতাটা একবারও পরিনি, সাইজ বড় হয়েছে। ফেরত দিতে চাই।"
  },
  "questions": {
    "allowed": {
      "type": "noul",
      "instructions": "Is this return allowed under `policy`?"
    },
    "action": {
      "type": "choice",
      "instructions": "What should the agent do next?",
      "criteria": {
        "approve_return": "accept the return",
        "ask_for_photos": "ask for photos before deciding",
        "reject": "refuse the return",
        "escalate": "send to a supervisor"
      }
    }
  }
}

Structured input

Read a whole conversation

Pass the chat as a list of turns and ask about the conversation as a whole.

Conversation
  1. customerAmar internet package kaj korche na.
  2. agentApni ki phone ta ekbar restart korechen?
  3. customerHaa, restart korar por thik hoye geche. Thanks!

Has the customer's problem been solved by the end of the chat? · yes / no

Nirnoy-flash
yes
99%
no
<1%
Nirnoy 12B
yes
99%
no
<1%

How does the customer feel at the end? · choice · 4 options

Nirnoy-flash
happy
94%
neutral
3%
annoyed
1%
angry
1%
Nirnoy 12B
happy
96%
neutral
2%
annoyed
1%
angry
1%

Nirnoy-flash: 68 ms · Nirnoy 12B: 57 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": [
    {
      "role": "customer",
      "text": "Amar internet package kaj korche na."
    },
    {
      "role": "agent",
      "text": "Apni ki phone ta ekbar restart korechen?"
    },
    {
      "role": "customer",
      "text": "Haa, restart korar por thik hoye geche. Thanks!"
    }
  ],
  "questions": {
    "solved": {
      "type": "noul",
      "instructions": "Has the customer's problem been solved by the end of the chat?"
    },
    "mood": {
      "type": "choice",
      "instructions": "How does the customer feel at the end?",
      "criteria": [
        "happy",
        "neutral",
        "annoyed",
        "angry"
      ]
    }
  }
}

Safety

Spot a scam message

Legitimate, spam or scam, with each option defined in Bangla.

Message

প্রিয় গ্রাহক, আপনার বিকাশ অ্যাকাউন্ট সাময়িকভাবে বন্ধ করা হয়েছে। চালু করতে এখনই আপনার পিন ও ওটিপি এই নম্বরে পাঠান।

এই বার্তাটি কি আসল, স্প্যাম, নাকি প্রতারণা? · choice · 3 options

Nirnoy-flash
legitimate
2%
spam
1%
scam
97%
Nirnoy 12B
legitimate
1%
spam
1%
scam
98%

Nirnoy-flash: 53 ms · Nirnoy 12B: 57 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "প্রিয় গ্রাহক, আপনার বিকাশ অ্যাকাউন্ট সাময়িকভাবে বন্ধ করা হয়েছে। চালু করতে এখনই আপনার পিন ও ওটিপি এই নম্বরে পাঠান।",
  "questions": {
    "scam": {
      "type": "choice",
      "instructions": "এই বার্তাটি কি আসল, স্প্যাম, নাকি প্রতারণা?",
      "criteria": {
        "legitimate": "আসল বার্তা: নোটিশ, ওটিপি, লেনদেন, সৎ অফার বা ব্যক্তিগত বার্তা",
        "spam": "অনাকাঙ্ক্ষিত প্রচার বা বিজ্ঞাপন যা ঠকায় না",
        "scam": "ঠকিয়ে টাকা, পিন, ওটিপি বা তথ্য নিতে চায়"
      }
    }
  }
}

Safety

Check a reply before it is sent

Give the policy and a draft reply, and ask whether the reply keeps to the policy.

Record (JSON)
{
  "question": "রিফান্ড পেতে কত দিন লাগে?",
  "policy": "রিফান্ড অনুমোদনের পর ৫ থেকে ৭ কর্মদিবসের মধ্যে টাকা ফেরত দেওয়া হয়।",
  "draft_reply": "চিন্তা করবেন না, আজকেই আপনার টাকা ফেরত চলে যাবে।"
}

Does `draft_reply` agree with `policy`? · yes / no

Nirnoy-flash
yes
1%
no
99%
Nirnoy 12B
yes
1%
no
99%

Does `draft_reply` promise something that `policy` does not allow? · yes / no

Nirnoy-flash
yes
98%
no
2%
Nirnoy 12B
yes
99%
no
1%

Nirnoy-flash: 68 ms · Nirnoy 12B: 56 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": {
    "question": "রিফান্ড পেতে কত দিন লাগে?",
    "policy": "রিফান্ড অনুমোদনের পর ৫ থেকে ৭ কর্মদিবসের মধ্যে টাকা ফেরত দেওয়া হয়।",
    "draft_reply": "চিন্তা করবেন না, আজকেই আপনার টাকা ফেরত চলে যাবে।"
  },
  "questions": {
    "consistent": {
      "type": "noul",
      "instructions": "Does `draft_reply` agree with `policy`?"
    },
    "overpromise": {
      "type": "noul",
      "instructions": "Does `draft_reply` promise something that `policy` does not allow?"
    }
  }
}

Understanding

Ask what a name refers to

Ask about each name in a message: a person, a place, a brand or a product.

Message

Walton er notun fridge ta kinlam, Daraz theke order korechilam. Ekdom thik moto kaj korche.

In this text, what kind of thing is "Walton"? · choice · 5 options

Nirnoy-flash
person
<1%
place
<1%
brand or company
98%
product
1%
other
<1%
Nirnoy 12B
person
1%
place
1%
brand or company
98%
product
1%
other
<1%

In this text, what kind of thing is "Daraz"? · choice · 5 options

Nirnoy-flash
person
<1%
place
2%
brand or company
95%
product
2%
other
1%
Nirnoy 12B
person
1%
place
1%
brand or company
96%
product
1%
other
1%

Nirnoy-flash: 67 ms · Nirnoy 12B: 56 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "Walton er notun fridge ta kinlam, Daraz theke order korechilam. Ekdom thik moto kaj korche.",
  "questions": {
    "walton": {
      "type": "choice",
      "instructions": "In this text, what kind of thing is \"Walton\"?",
      "criteria": [
        "person",
        "place",
        "brand or company",
        "product",
        "other"
      ]
    },
    "daraz": {
      "type": "choice",
      "instructions": "In this text, what kind of thing is \"Daraz\"?",
      "criteria": [
        "person",
        "place",
        "brand or company",
        "product",
        "other"
      ]
    }
  }
}

Languages

Bangla, Banglish or English

The same question on the same message written three ways. No language setting, no transliteration step.

Message · Bangla

প্রোডাক্টটা খুব ভালো, তবে ডেলিভারি দেরিতে এসেছে।

What is the overall sentiment of this review? · choice · 4 options

Nirnoy-flash
positive
1%
negative
1%
neutral
1%
mixed
97%
Nirnoy 12B
positive
2%
negative
1%
neutral
1%
mixed
96%

Nirnoy-flash: 53 ms · Nirnoy 12B: 49 ms for the whole request

Message · Banglish

Product ta khub valo, tobe delivery deri te asche.

What is the overall sentiment of this review? · choice · 4 options

Nirnoy-flash
positive
1%
negative
1%
neutral
1%
mixed
97%
Nirnoy 12B
positive
2%
negative
1%
neutral
2%
mixed
95%

Nirnoy-flash: 53 ms · Nirnoy 12B: 49 ms for the whole request

Message · English

The product is great, but the delivery came late.

What is the overall sentiment of this review? · choice · 4 options

Nirnoy-flash
positive
1%
negative
1%
neutral
1%
mixed
97%
Nirnoy 12B
positive
2%
negative
1%
neutral
1%
mixed
96%

Nirnoy-flash: 47 ms · Nirnoy 12B: 49 ms for the whole request

The request as JSON
POST /v1/systemone
{
  "state": "প্রোডাক্টটা খুব ভালো, তবে ডেলিভারি দেরিতে এসেছে।",
  "questions": {
    "sentiment": {
      "type": "choice",
      "instructions": "What is the overall sentiment of this review?",
      "criteria": [
        "positive",
        "negative",
        "neutral",
        "mixed"
      ]
    }
  }
}
POST /v1/systemone
{
  "state": "Product ta khub valo, tobe delivery deri te asche.",
  "questions": {
    "sentiment": {
      "type": "choice",
      "instructions": "What is the overall sentiment of this review?",
      "criteria": [
        "positive",
        "negative",
        "neutral",
        "mixed"
      ]
    }
  }
}
POST /v1/systemone
{
  "state": "The product is great, but the delivery came late.",
  "questions": {
    "sentiment": {
      "type": "choice",
      "instructions": "What is the overall sentiment of this review?",
      "criteria": [
        "positive",
        "negative",
        "neutral",
        "mixed"
      ]
    }
  }
}

About these answers

Each request went to the same server code as production (the /v1/systemone API), one question per row, so adding or removing a question never changes another answer. The time is for the whole request on one GPU, after a warm-up.

For a choice question with many options, the four most likely are shown. The probabilities track how often Nirnoy is right: on our human-labelled test, Nirnoy-flash's answers at 90% or more (56% of all answers) were right 92% of the time, and its answers below 60% only 43%. So you can pick a threshold and send the rest to a person.