What is JsonValidityEvaluator in Langchain

The JsonSchemaEvaluator validates a JSON prediction against a provided JSON schema. If the prediction conforms to the schema, it returns a score of True (indicating no errors). Otherwise, it returns a score of 0 (indicating an error).

def json_validity_evaluation():

    evaluator = JsonValidityEvaluator()

    # Equivalently

    # evaluator = load_evaluator("json_validity")

    prediction = '{"name": "John", "age": 30, "city": "New York"}'

    result = evaluator.evaluate_strings(prediction=prediction)

    print("Score 1 is ", result)

    prediction = '{"name": "John", "age": 30, "city": "New York",}'

    result = evaluator.evaluate_strings(prediction=prediction)

    print("Score 2 is ", result)

    result = evaluator.evaluate_strings(

    prediction='{"name": "John", "age": 30}',

    reference='{"type": "object", "properties": {"name": {"type": "string"}, "age": {"type": "integer"}}}',

    )

    print(result)

References:

https://python.langchain.com/v0.1/docs/guides/productionization/evaluation/string/json/