Form Validation Webhook

A form validation webhook allows you to plug in your own code to validate the information entered on a form.

What's a Form Validation Webhook?

When someone submits a form, FormSmarts validates their input. If we find the person provided incorrect information, we asks them to modify their entry. Form validation identifies errors like syntactically incorrect phone numbers and invalid email addresses and helps prevent typing mistakes.

For example, FormSmarts would reject lucy@@gmail.com because the format of the address is invalid and lucy@gmail.co because no email servers are associated with gmail.co.

Standard form validation is based on universal rules (Internet standards), but sometimes you also need to validate data fields based on rules that only apply to a specific application within your organization. You might for example need to check that the serial number submitted on a product registration form exists and that the product hasn't already been registered. For other applications like online voting, you may need to verify a password and record the ID of people who have already voted in a database.

FormSmarts supports these applications by allowing customers¹ to execute custom computer code when a form is submitted and authorize or not the form submission to proceed.

A validation webhook works as follows:

1. A user submits a form
2. FormSmarts sends an HTTP request to the webhook URL set up by the form owner
3. The request launches a program that queries/updates a database etc. to validate the data
4. The program returns a valid or invalid response to FormSmarts, with a list of errors if applicable
5. The user can proceed if the response is valid, otherwise they're invited to modify their input and resubmit (back to #2).

Here are a few applications of validation hooks:

Python Example: Online Election

For demonstration purposes, we've created a web application that allows anyone with an email address to vote online. Please feel free to try it here.

• The web app verifies the voter's email using FormSmarts email address and phone number verification system
• We wrote a validation webhook in Python (complete code below) and deployed it to a code execution environment (AWS Lambda)
• We set up another webhook to run the code again and change the status of the vote to confirmed when the voter confirms their choice (they could change their election choices up to this point)
• We send a confirmation email using FormSmarts autoresponder.

Details of the code are not important, but note how we:

• Add the email of the voter to a database if they haven't already voted
• Return {'valid': True} if the vote was accepted, and an error message (shown to the user) otherwise:

REJECT_RESP = {
    'valid': False,
    'invalid_fields': {
        EMAIL_FIELD_ID: {
            'message': 'You have already voted.'
        },
    },
}

The Code

We built this demo on the AWS cloud. In addition to Lambda, a code execution service, we use DynamoDB, a database service and API Gateway, that runs the code when FormSmarts sends an HTTP request.

Instead of AWS, we could have used Cloudflare Workers and Workers KV or offers from Google Cloud or Microsoft Azure.

Although serverless cloud platforms provide a reliable and scalable environment for hosting webhooks, you can of course use technology you are already familiar with and is readily available, for example PHP and a MySQL database on your website.

import os
import time
import json
import boto3
import webhook_authenticator
from datetime import datetime
from botocore.exceptions import ClientError
from boto3.dynamodb.conditions import Attr, Or

EMAIL_FIELD_ID = 702283
ACCEPT_RESP = {'valid': True}
REJECT_RESP = {
    'valid': False,
    'invalid_fields': {
        EMAIL_FIELD_ID: {
            'message': 'You have already voted.'
        },
    },
}

def lambda_handler(event, context):
    msg = event['body']
    # Verify request is from FormSmarts
    auth = webhook_authenticator.Authenticator(
        os.environ['FORMSMARTS_WEBHOOK_KEY'], os.environ['WEBHOOK_URL'], msg
    )
    if auth.verify_request(event['headers']['authorization']):
        msg = json.loads(msg)
        fields = {fd['field_id']: fd['field_value'] for fd in msg['fields']}
        email = fields[EMAIL_FIELD_ID]
        db = boto3.resource(
            'dynamodb',
            region_name=os.environ['AWS_REGION']
        ).Table('election_demo')
        if msg.get('action') == 'validate':
            # This is validation webhook, record vote if the person hasn't already voted
            try:
                db.put_item(
                    Item=dict(
                        email=email,
                        status='pending',
                        ttl=int(time.time()) + 2 * 3600,  # Keep pending votes for 2 hours
                        txid=msg['fs_ref_num'],
                        timestamp=datetime.utcnow().isoformat()
                    ),
                    ConditionExpression=Or(
                        Attr('email').not_exists(), Attr('txid').eq(msg['fs_ref_num'])
                    )
                )
            except ClientError as err:
                if err.response['Error']['Code'] == 'ConditionalCheckFailedException':
                    resp = REJECT_RESP
                else:
                    raise
            else:
                resp = ACCEPT_RESP
            return {'statusCode': 200, 'body': json.dumps(resp)}
        else:
            # This is a submit webhook, confirm vote
            ttl = int(time.time()) + 90 * 24 * 3600  # Keep voting data for 3 months
            db.update_item(
                Key=dict(email=email),
                UpdateExpression='SET #status = :s, #ttl = :ttl',
                ExpressionAttributeNames={'#status': 'status', '#ttl': 'ttl'},
                ExpressionAttributeValues={':s': 'confirmed', ':ttl': ttl},
                ReturnValues='NONE',
            )
            return {'statusCode': 200, 'body': json.dumps(dict(message='Ok'))}
    else:
        return {
            'statusCode': 401,
            'body': json.dumps('Invalid webhook signature.')
        }

Authenticate Requests

When you create a webhook, you should always authenticate the request as originating from FormSmarts before taking any actions based on the data in the request.

The code above verifies requests with FormSmarts Webhook Authenticator for Python. We also provide tools or alternative options for other languages.

Webhook authentication relies on two tokens, your FormSmarts account ID and your secret webhook key.

You'll find your Account ID in the Account Overview section of your account and your Webhook Key in the Security Settings.

Find Input Field IDs

It is sometimes convenient to hardcode the ID of an input field in the code, as we do with EMAIL_FIELD_ID = 702283 above.

To find the ID of input fields of a form, log in to the API Console and submit a GET request to the API endpoint https://formsmarts.com/api/v1/forms/Form_ID/fields, as described in the Form API documentation.

For example, the screenshot below shows the ID (right-most column) of each of the fields of one of our demos.

Register the Webhook URL

Once you've written the webhook code you want to execute to validate form responses, you need to register its URL with FormSmarts via the Web API. The API also allows you to view the current webhook URL and cancel the webhook. You can call the API with the API Console, curl or programmatically.

There is an example showing how to do this with the API Console in the submit webhook documentation (remember to use the API endpoint below instead of the one shown on the screenshot, which is for a different type of webhook).

From Validation Webhook Automation Python

1. Not available with Business Starter and Plus accounts.