Skip to main content

Specialized Functions

Vadalog provides a comprehensive library of specialized functions for mathematical operations, hashing, date/time manipulation, type casting, and interval checks.

Mathematical Functions​

math:mod()​

Modulo operation:

math:mod(dividend: number, divisor: number) → number

Example:

is_even(N) <- 
    numbers(N), 
    Remainder = math:mod(N, 2), 
    Remainder == 0.

math:sqrt()​

Square root:

math:sqrt(x: number) → double

Example:

distance(X, Y, Distance) <- 
    point(X, Y), 
    Distance = math:sqrt(X * X + Y * Y).

math:abs()​

Absolute value:

math:abs(x: number) → number

Example:

price_difference(Item1, Item2, Diff) <- 
    price(Item1, P1), 
    price(Item2, P2), 
    Diff = math:abs(P1 - P2).

math:round()​

Round to nearest integer or specified decimal places:

math:round(x: number) → number
math:round(x: number, scale: int) → number

Example:

rounded_price(Item, RoundedPrice) <- 
    price(Item, RawPrice), 
    RoundedPrice = math:round(RawPrice, 2).

math:bround()​

Banker's rounding (round to nearest even):

math:bround(x: number) → number
math:bround(x: number, scale: int) → number

Example:

banker_rounded(Value, Rounded) <- 
    data(Value), 
    Rounded = math:bround(Value, 2).

math:min() / math:max()​

Find minimum or maximum value:

math:min(v1: comparable, v2: comparable, …) → same type
math:max(v1: comparable, v2: comparable, …) → same type

Example:

price_range(Product, MinPrice, MaxPrice) <- 
    prices(Product, P1, P2, P3, P4), 
    MinPrice = math:min(P1, P2, P3, P4), 
    MaxPrice = math:max(P1, P2, P3, P4).

math:log()​

Natural logarithm:

math:log(x: number) → double

Example:

log_value(X, LogX) <- 
    data(X), 
    LogX = math:log(X).

math:log10()​

Base-10 logarithm:

math:log10(x: number) → double

Example:

magnitude(Value, Magnitude) <- 
    measurement(Value), 
    Magnitude = math:log10(Value).

math:pow()​

Power (exponentiation):

math:pow(base: number, exponent: number) → double

Example:

compound_interest(Principal, Rate, Time, Amount) <- 
    investment(Principal, Rate, Time), 
    Amount = Principal * math:pow(1 + Rate, Time).

math:exp()​

Exponential (e^x):

math:exp(x: number) → double

Example:

exponential_growth(Rate, Time, Growth) <- 
    parameters(Rate, Time), 
    Growth = math:exp(Rate * Time).

math:ceil() / math:floor()​

Round up or down to nearest integer:

math:ceil(x: number) → number
math:floor(x: number) → number

Example:

quantity_boxes(Items, Boxes) <- 
    order(Items), 
    Boxes = math:ceil(Items / 24).  % 24 items per box

math:sin() / math:cos() / math:tan()​

Trigonometric functions:

math:sin(x: number) → double
math:cos(x: number) → double
math:tan(x: number) → double

Example:

wave_value(Angle, SineWave) <- 
    angles(Angle), 
    SineWave = math:sin(Angle).

math:uniform() / math:rand()​

Generate random numbers:

math:uniform(upperBound: number) → number       % Random integer [0, upperBound)
math:rand() → double                            % Random double [0, 1)

Example:

random_customer_id(ID) <- 
    customer(Name), 
    ID = math:uniform(100000).

math:PI() / math:E()​

Mathematical constants:

math:PI() → double
math:E() → double

Example:

circle_area(Radius, Area) <- 
    circle(Radius), 
    Area = math:PI() * math:pow(Radius, 2).

Hash Functions​

Generate hash values for data integrity and security.

hash:hash()​

Generate 32-bit Spark hash:

hash:hash(v1: any, v2: any, …) → int

Example:

record_hash(Name, Hash) <- 
    person(Name, Age, Email), 
    Hash = hash:hash(Name, Age, Email).

hash:sha1()​

Generate SHA-1 hash:

hash:sha1(value: any) → string

Example:

secure_user_sha1(Username, PasswordHash) <- 
    user(Username, Password), 
    PasswordHash = hash:sha1(Password).

hash:md5()​

Generate MD5 hash:

hash:md5(value: any) → string

Example:

secure_user_md5(Username, PasswordHash) <- 
    user(Username, Password), 
    PasswordHash = hash:md5(Password).

hash:sha2()​

Generate SHA-2 hash with specified bit length:

hash:sha2(v1: any, …, bitLength: int) → string

Examples:

% SHA-256 hash
secure_user_sha256(Username, PasswordHash) <- 
    user(Username, Password), 
    PasswordHash = hash:sha2(Password, 256).

% SHA-2 with multiple values (composite keys)
composite_hash(User, Pass, Hash) <- 
    credentials(User, Pass), 
    Hash = hash:sha2(User, Pass, 256).

Date and Time Functions​

date:current_date() / date:current_timestamp()​

Get current date or timestamp:

date:current_date() → date
date:current_timestamp() → timestamp

Example:

record_created(ID, CreatedAt) <- 
    new_record(ID), 
    CreatedAt = date:current_timestamp().

date:next_day() / date:prev_day()​

Add or subtract one day:

date:next_day(date: date) → date
date:prev_day(date: date) → date

Example:

due_date(Task, DueDate) <- 
    task(Task, StartDate), 
    DueDate = date:next_day(StartDate).

date:add() / date:sub()​

Add or subtract days:

date:add(date: date, days: int) → date
date:sub(date: date, days: int) → date

Example:

expiry_date(Product, ExpiryDate) <- 
    product(Product, ManufactureDate, ShelfLife), 
    ExpiryDate = date:add(ManufactureDate, ShelfLife).

date:diff()​

Calculate difference in days:

date:diff(end: date, start: date) → int

Example:

employee_tenure(Name, Years) <- 
    employee(Name, HireDate), 
    Today = date:current_date(), 
    Days = date:diff(Today, HireDate), 
    Years = Days / 365.

date:spec_day()​

Parse date with specific format:

date:spec_day(text: string "MMM-yyyy") → string

Example:

formatted_date(InvestmentDate, FormattedDate) <- 
    investment(InvestmentDate), 
    FormattedDate = date:spec_day(InvestmentDate).

date:to_timestamp()​

Parse string to timestamp:

date:to_timestamp(text: string, format: string) → timestamp

Example:

parsed_timestamp(Raw, Ts) <- 
    raw_date(Raw), 
    Ts = date:to_timestamp(Raw, "yy-M-dd HH:mm:ss,SSS Z").

date:format()​

Format date or timestamp to string:

date:format(dateOrTs: date|timestamp, pattern: string) → string

Example:

formatted_report(Name, FormattedDate) <- 
    event(Name, EventDate), 
    FormattedDate = date:format(EventDate, "yyyy-MM-dd").

Type Casting Functions​

Convert values between different data types.

as_string()​

Cast to string:

as_string(value: any) → string

Example:

formatted_id(ID) <- 
    customer(NumericID), 
    ID = concat("CUST-", as_string(NumericID)).

as_int() / as_long()​

Cast to integer or long:

as_int(value: any) → int
as_long(value: any) → long

Example:

numeric_value(Value) <- 
    raw_data(StringValue), 
    Value = as_int(StringValue).

as_double() / as_float()​

Cast to double or float:

as_double(value: any) → double
as_float(value: any) → float

Example:

precise_value(Value) <- 
    raw_data(StringValue), 
    Value = as_double(StringValue).

as_boolean()​

Cast to boolean:

as_boolean(value: any) → boolean

Example:

is_active(Name, Active) <- 
    user(Name, StatusString), 
    Active = as_boolean(StatusString).

as_date() / as_timestamp()​

Cast to date or timestamp:

as_date(value: string|timestamp) → date
as_timestamp(value: string) → timestamp

Example:

event_date(EventName, ParsedDate) <- 
    event(EventName, DateString), 
    Timestamp = date:to_timestamp(DateString, "dd/MM/yyyy"),
    ParsedDate = as_date(Timestamp).

as_json()​

Cast to JSON string:

as_json(value: any) → string

Example:

json_output(Data, JsonString) <- 
    structured_data(Data), 
    JsonString = as_json(Data).

as_list()​

Parse a JSON array string into a typed Spark array. Accepts two forms:

as_list(json_string: string, element_type: string) → array
as_list(json_string: string, "field1:type1, field2:type2, ...") → array<struct>

When the second argument is a simple type (e.g., "string", "integer"), the JSON array elements are cast to that type. When it contains : characters, it is parsed as a struct schema — each JSON object in the array is parsed into a struct with the declared fields.

Simple type example:

data("[1, 2, 3, 4]").
result(L) <- data(J), L = as_list(J, "integer").
% L is an array of integers: [1, 2, 3, 4]

Struct schema example (e.g., Qdrant ask() results):

result(Name, Score) <- source(Query),
    J = ask("find ${Query}", "collection=my_collection,limit=3"),
    L = as_list(J, "name:string, category:string, __score__:double"),
    First = collections:get(L, 1),
    Name = struct:get("name", First),
    Score = struct:get("__score__", First).

Schema rules:

  • Fields are matched by name, not by position — the order of fields in the schema does not need to match the JSON.
  • You only need to declare the fields you want to extract. Omitted fields are ignored.
  • Supported types: string, double, integer, long, float, boolean, date, timestamp.

as_struct()​

Parse a single JSON object string into a typed Spark struct:

as_struct(json_string: string, "field1:type1, field2:type2, ...") → struct

Same schema format as as_list, but for a single JSON object (not an array). Returns a struct directly.

Example:

data("{""name"":""Alice"",""age"":30,""city"":""London""}").

result(Name, Age) <- data(J),
    S = as_struct(J, "name:string, age:integer"),
    Name = struct:get("name", S),
    Age = struct:get("age", S).
@output("result").
% Result: ("Alice", 30)
% Note: "city" was in the JSON but not in the schema — it is simply ignored.
as_list vs as_struct

Use as_list when the JSON is an array of objects (e.g., [{...}, {...}]). Use as_struct when the JSON is a single object (e.g., {...}). Both use the same "field:type, ..." schema format and both match fields by name.

as_set()​

Parse a JSON array string into a typed Spark array with duplicates removed:

as_set(json_string: string, element_type: string) → array (distinct)

Example:

data("[1, 2, 2, 3, 3, 3]").
result(S) <- data(J), S = as_set(J, "integer").
% S is [1, 2, 3]

as_map()​

Cast to a map type:

as_map(value: any, key_type: string, value_type: string) → map

Interval Operators​

Check if values fall within specified ranges.

between()​

Exclusive on both ends:

between(value, lower, upper) → boolean

Example:

strict_range(X) <- 
    data(X), 
    InRange = between(X, 5, 10), 
    InRange == #T.
% Returns true for 6, 7, 8, 9 (not 5 or 10)

_between()​

Inclusive left, exclusive right:

_between(value, lower, upper) → boolean

Example:

left_inclusive(X) <- 
    data(X), 
    InRange = _between(X, 5, 10), 
    InRange == #T.
% Returns true for 5, 6, 7, 8, 9 (not 10)

between_()​

Exclusive left, inclusive right:

between_(value, lower, upper) → boolean

Example:

right_inclusive(X) <- 
    data(X), 
    InRange = between_(X, 5, 10), 
    InRange == #T.
% Returns true for 6, 7, 8, 9, 10 (not 5)

_between_()​

Inclusive on both ends:

_between_(value, lower, upper) → boolean

Example:

fully_inclusive(X) <- 
    data(X), 
    InRange = _between_(X, 5, 10), 
    InRange == #T.
% Returns true for 5, 6, 7, 8, 9, 10

% Practical example: filter customers by age range
adult_customers(Name, Age) <- 
    customer(Name, Age), 
    IsAdult = _between_(Age, 18, 65), 
    IsAdult == #T.

Null Handling Functions​

is_null()​

Check if a value is null:

is_null(expression: any) → boolean

Example:

@output("b").
a("hello").
b(El, State) <- a(El), State = is_null(El).

Expected output:

b("hello", #F).

is_not_null()​

Check if a value is not null:

is_not_null(expression: any) → boolean

Example:

@output("b").
a("hello").
b(El, State) <- a(El), State = is_not_null(El).

Expected output:

b("hello", #T).

nullManagement:ifnull()​

Return a fallback value when an expression is null, or a different value when it is not null:

nullManagement:ifnull(expression: any, valueIfNull: any, valueIfNotNull: any) → any

Example:

@output("b").
a("hello").
b(El, Result) <- a(El), Result = nullManagement:ifnull(El, "was_null", "was_not_null").

Expected output:

b("hello", "was_not_null").

nullManagement:coalesce()​

Return the first non-null value from a list of expressions:

nullManagement:coalesce(expr1: any, expr2: any, ...) → any

Example:

@output("b").
a("hello").
b(Result) <- a(El), Result = nullManagement:coalesce(El, "fallback").

Expected output:

b("hello").

Utility Functions​

struct()​

Create structured data by pairing keys with values:

struct(key1: string, value1: any, key2: string, value2: any, ...) → struct

Example:

record_with_metadata(X, S) <- 
    data(X), 
    S = struct("originalValue", X, "incrementedValue", X + 1).

struct:get()​

Extract a field value from a struct by its field name:

struct:get(fieldName: string, struct: struct) → any

Example:

a(10, 20).
b(S) <- a(X, Y), S = struct("field1", X, "field2", Y).
c(F1, F2) <- b(S), F1 = struct:get("field1", S), F2 = struct:get("field2", S).
@output("c").

Expected output:

c(10, 20).

uuid()​

Generate universally unique identifiers:

uuid() → string                                 % Random UUID
uuid(arg1: any, arg2: any, ...) → string       % Deterministic UUID

Examples:

% Generate random UUIDs
random_id(A, U) <- 
    input(A), 
    U = uuid().

% Generate deterministic UUIDs for consistent identification
patient_id(Clinic, Patient, UniqueID) <- 
    patient_data(Clinic, Patient), 
    UniqueID = uuid(Clinic, Patient).

monotonically_increasing_id()​

Generate monotonically increasing IDs:

monotonically_increasing_id() → long

Example:

numbered_records(Name, ID) <- 
    records(Name), 
    ID = monotonically_increasing_id().

utils:phone_number()​

Format phone numbers with optional regional context:

utils:phone_number(phone: string) → string
utils:phone_number(phone: string, region: string) → string

Examples:

% Format phone number with default region
formatted_phone(Ph, Fmt) <- 
    phone_data(Ph), 
    Fmt = utils:phone_number(Ph).

% Format with specific region
formatted_phone_uk(Ph, Fmt) <- 
    phone_data(Ph), 
    Fmt = utils:phone_number(Ph, "GB").

Anonymization Functions​

Privacy-preserving data operations.

mask()​

Mask sensitive data for privacy protection:

mask(input: string) → string
mask(input: string, upperChar: char, lowerChar: char, digitChar: char, otherChar: char) → string

Masks the given string value by replacing characters with configurable mask characters. The function replaces characters with 'X' or 'x', and numbers with 'n' by default. This is useful for creating copies of tables with sensitive information removed.

Arguments:

  • input - string value to mask (STRING, VARCHAR, CHAR)
  • upperChar - character to replace upper-case characters with. Specify NULL to retain original character. Default: 'X'
  • lowerChar - character to replace lower-case characters with. Specify NULL to retain original character. Default: 'x'
  • digitChar - character to replace digit characters with. Specify NULL to retain original character. Default: 'n'
  • otherChar - character to replace other characters with. Specify NULL to retain original character

Examples:

% Basic masking with default characters
masked_email(Name, MaskedEmail) <- 
    user(Name, Email), 
    MaskedEmail = mask(Email).
% Input: "John.Doe123@example.com" 
% Output: "Xxxx.Xxx nnn@xxxxxxx.xxx"

% Custom masking with specific characters
custom_masked_ssn(Person, MaskedSSN) <- 
    person(Person, SSN), 
    MaskedSSN = mask(SSN, '*', '*', '#', '-').
% Input: "123-45-6789"
% Output: "###-##-####"

% Preserve certain character types
partial_mask(Data, PartiallyMasked) <- 
    sensitive_data(Data), 
    PartiallyMasked = mask(Data, 'X', NULL, '#', NULL).
% Masks only uppercase letters and digits, preserves lowercase and symbols

anonymization:risk()​

Calculate privacy risk score:

anonymization:risk(sampleFrequency: number, weights: array<number>, nIterations: int) → double

Example:

privacy_assessment(Dataset, RiskScore) <- 
    sensitive_data(Dataset, Attributes), 
    RiskScore = anonymization:risk(0.1, [1.0, 0.8, 0.6], 1000).

anonymization:msu()​

Calculate MSU score:

anonymization:msu(attributes: any, m: int, k: int) → int

Example:

msu_score(Data, Score) <- 
    dataset(Data, Attrs), 
    Score = anonymization:msu(Attrs, 5, 3).

Kalman Filter Functions​

Advanced signal processing and fault detection.

kalmanFilter:fault()​

Detect faults using Kalman filter:

kalmanFilter:fault(A: any, B: any, C: any, R: any, Q: any, X: any, U: any, nu: any) → string

kalmanFilter:extractor()​

Extract values from Kalman filter input:

kalmanFilter:extractor(input1: any, input2: any, inputNumber: int) → int