Parse, Don't Validate (2019)

When you have a language with a strong type system, this is one of the practical things that ultimately gives you freedom as your program gets bigger and more complex.

But you have to use it. E.g. by having a Class UncheckedEmail, a Class ValidEmail and a Class VerifiedEmail and ensuring that the conversion from one to the other has to involve your email-verification process.

That way you never have to guess whether the email adress is unchecked, valid or verified and there is no need for "is_email_verified" booleans that you may or may not forget to update/check. If you use the wrong thing in the wrong place your type checker yells at you, while you can focus on actually important stuff.

is seeing already guaranteed properties being checked again somewhere else

Because at some point those "already guaranteed properties" might "disappear", to be more exact, the process/procedure that implements and runs them might not do its thing anymore for one reason or another.

When that happens, because statistically speaking it will happen, then all the other processes/scripts/pieces of code depending on that "original" validating process would be in a very rough place.

I've been going through the comments on previous posts. I think one of the biggest issues with the article is the title. It seems to act as an anchor for a lot of people to the point where they'll argue against things that aren't in the article, just implied by the title (without context). So there'll be people arguing that she doesn't want to validate at all and just wants to parse, when really the article is about where you validate your data (and what you do with the result). It is not about getting rid of all validation.

The definition of a list is that it contains zero or more elements. There's nothing deep about it; it's just the way the type is defined (and always has been, back to the early days of Lisp from which functional programming evolved.)

Haskell has this simply for historical reasons. It's a wart and we wish it weren't so but changing it would break a lot of legacy code (and probably have adverse performance impacts in some cases) so it remains. So as it is today, [a] indeed means it can be empty or not. And a function signature of "[a] -> a" is essentially an unsafe partial function.

That isn't to say you _have_ to write it this way for new things and there are packages like https://hackage.haskell.org/package/safe which provide non-partial/safe versions of the various unsafe base functions (like head, maximum and friends).

The base package in Haskell also includes nonempty which is probably what you want in a lot of cases, anyway.

The type of elements and the number of elements are two orthogonal aspects. You might need at least one element, or you might need at least two elements, or you might need an even number of elements, and so on. This is independent from whether you need the elements to be of type a (or some other condition on the element types). Ideally, all these conditions could be expressed in the type system. But the requirement "all elements must be of type a" by itself does not imply "there must be at least one element", nor does it necessarily imply having to special-case the empty list.

There's no reason this couldn't be done. Indeed, in OCaml (which I am more familiar with), you could easily define:

```ocaml

type 'a nonempty = Single of 'a | Cons of 'a * 'a nonempty

```

This would be the type of lists that contain one or more elements of the type parameter.

I think it's just convention that typically, when we talk about lists, we are interested in the empty case as well. Finding "all X that satisfy P in Y", as a general computational problem, is _very_ common (consider: filtering a list, querying for a predicate in a collection, finding sequences of moves in a search space), and generally could result in an empty list as a possible output.

In a non-practical sense, if you want the type theory, another reason is you can think of `[a]` as the free monoid on the collection of `a`. In other words, strings of elements of `a`, joined via concatenation. This monoid requires a unit, which is the empty list.

(2019), but still good-ish advice

Why only "good-ish"? And how does it relate to the year the article was published? Surely you are implying that the advice in the article would be more authoritative if it were published earlier than 2019, right?

Protip: Don't do either. And definitely don't do the first.

Explicit is always better than implicit defaults that get used instead when you give it a wrong value that you think is correct.

What you should do is throw your hands up early, fail to parse, and have a very clearly defined process and protocol to handle files that couldn't be loaded. It'll force you to ask yourself very difficult questions that aren't covered by either of the two options you posted.

The real failure in the recent Crowdstrike kernel-mode driver failing to parse some def/config file is that the dev/product owner/BA didn't ask "what happens if we try load a file that's invalid?"

Please don't do the first. Handle the bad cases. "sane default" fallback should be extremely rare.

Explicit > Implicit

if(!Whatever.TryParse<Thingy>(input, out var output)) output = some-sane-default;

I can't think of many (probably any) situations where I'd want to find that.

If _no_ input is provided (i.e., the parameter is optional), sure, using a sane default makes sense.

If _invalid_ input is provided, for the love of god please don't pretend like nothing's wrong.

If someone walks into a florist and asks for a coffee, the correct answer is not for them to be handed a rose. They're going to cut their mouth all up when they try and drink it.

Your method/module/program does not have an output defined for that set of inputs. Make that obvious rather than just doing wrong or non-obvious things in a way that quickly makes your program almost impossible to reason about. Do yourself a favour and clearly raise the issue and leave yourself a stack trace pointing directly to the issue instead of setting yourself up for the vague bug about incorrect behaviour when someone catches this in a few months.

if(!Whatever.TryParse<Thingy>(input, out var output)) output = some-sane-default;

I absolutely hate that. IMO you should handle the error of an invalid input outside of the function to parse. F# makes that easy.

    type Whatever =
    static member create input =
      match input with
      | ValidWhatever x -> Some x
      | _ -> None

    match Whatever.create input with
    | Some x -> //process the parsed data
    | None -> //handle it not being parsed well

With this, you can only instantiate any instances through the create method with parses the input.

Altough, you probably want to use a result rather than option, but I digress.

No, the idea is that your application level validation functions take in data, own it and return a validation error or new type that has invalid states removed.

In this case you could make a wrapper function that accepts the raw binary data, passes it to Cap'n'Proto, validates the output (this field is actually required etc) and then returns it

The problem here as I understood it is more general and orthogonal to what the Parse, Don't Validate article is talking about.

1. A field being "required", is not a property of the field itself, but of the construct holding the field. JSON-Schema does this correctly, by letting you define an array of required fields on the schema of an object instead of it being a property of a specific field.

2. Consumers, not producers should decide what their assumptions are. Producers should decide what their guarantees are.

3. Everyone, including the parts in the middle (here it's a message bus) should only state assumptions that they actually need in order to function. Use different schemas for different assumptions (which is easier to do if the "required" assumption is a property of a construct and not a field).

The example in the article you mentioned illustrates nicely how these three principles are broken by "required" (or maybe how Protocol Buffers are used in general).

I imagine it can be made to work, as long as you make sure the zero value is never actually valid; then you're back in the usual billion-dollar-mistake world where everything is implicitly optional, rather than in the hellscape where you can never know whether you've got something meaningful in your hand or not. (I realise that Golang strongly believes zero values are usually valid, but Golang is wrong about this.)

`Void` is not equivalent to the unit type. Haskell _has_ a unit type, called `()`. A return value of `Void` represents a function which is unimplementable, or which never terminates. It can also represent an unreachable possibility, e.g. in the type `Either Void a`.

This is a super tool! But it still requires one to write good gates.

For all projects of some. Size I would advice people to use Zod or the like (unless there are special circumstances such that external deps can not be used).

Parsing subsumes validation.

You can make your own types.

Obviously if you've written a buggy parser then all bets are off, just as all bets are off if you've written bugs into any other part of your program. But those bugs would probably still be there if you didn't explicitly parse the input - they'd just appear later on, mixed in with your business logic which is performing ad-hoc parses whenever it needs the data!

Not really the same thing.

The point is that validation alone is then lost as information later on.

E.g. validating an int to be positive has limited benefits if you don't parse it to be a positive int, because there's no such information at the type level later on, same could apply to a non empty array/list where following consumers would then need to check again if the list is really non empty.

This kind of information cannot always be encoded in objects or constructors.

I'm going to make the bold claim that it doesn't take more time to do this, if your language supports algebraic data types. It just sort of naturally happens. Of course, if your language requires a great deal of ceremony (C++, Java, C#, Python, Golang, Javascript, …) to model data, then it will take more time.