The FormValidator class is accessible via the formencode.validators module.
The important method is .validate(), of course. It gets passed a dictionary of the (processed) values from the form. If you have .validate_partial_form set to True, then it will get the incomplete values as well -- use .has_key() to test if the field was able to process any particular field.
Anyway, .validate() should return a string or a dictionary. If a string, it's an error message that applies to the whole form. If not, then it should be a dictionary of fieldName: errorMessage. The special key "form" is the error message for the form as a whole (i.e., a string is equivalent to {"form": string}).
Return None on no errors.
Messages
See the source for more information.