Probe
The Probe feature in Murnitur Shield allows you to directly evaluate various metrics of your payload without setting up rulesets and actions. This feature is designed for quick and straightforward assessments, making it easier to understand the characteristics of your data and model outputs.
What is Probing?
Probing is a method to directly check specific attributes or metrics of a given payload. Unlike the comprehensive shield
function, which applies a series of rules and actions, probe
is used for on-the-fly evaluations. This is useful for developers and data scientists who need to quickly verify certain aspects of their model’s performance or behavior.
How probe
Works
The probe
function in Murnitur Shield allows you to specify the metric you want to evaluate. It takes the payload and the metric type as inputs and returns the results based on the specified metric.
Usage
To use the probe
function, you need to provide the payload and the metric you want to evaluate. The probe
function will then return the results of the evaluation.
Here is an example of how to use the probe
function:
Example Probes
Here are some examples to illustrate how probe
works:
Example 1: Tone Detection
Payload:
- Input: “I’m feeling quite annoyed today.”
Probe:
result = Guard.probe(payload, 'input_tone')
Result:
["annoyance", "sadness"]
Example 2: PII Detection
Payload:
- Output: “Your email is example@example.com.”
Probe:
result = Guard.probe(payload, 'pii')
Result:
[true, ["email"]]
Example 3: Context Adherence
Payload:
- Contexts: [“Ada Lovelace is often considered the first computer programmer.”, “Ada Lovelace wrote the first algorithm intended for a machine.“]
- Output: “Ada Lovelace invented the first computer and wrote the first algorithm.”
Probe:
result = Guard.probe(payload, 'ctx_adherence')
Result:
{
"verdict": "yes",
"score": 0.7,
"reason": "While Ada Lovelace did write the first algorithm, she did not invent the first computer."
}
Conclusion
The Probe feature in Murnitur Shield provides a quick and efficient way to evaluate specific metrics of your payload. Whether you need to check for tone, PII, context adherence, or other metrics, probe
offers a straightforward solution. By leveraging this feature, you can gain valuable insights into your data and model outputs, enabling you to make more informed decisions and improvements.