Start by preparing the fritter batter. In a bowl, whisk together the egg, milk, flour, baking powder, salt, and pepper.
Heat a large skillet over medium heat and add enough oil to cover the bottom of the pan.
Dip the whitebait into the batter and carefully place them into the hot oil. Fry for approximately 3 minutes, or until golden brown.
Remove the fritters from the pan and place them on a paper towel-lined plate to absorb any excess oil.
To make the tartare sauce, simply mix together the mayonnaise, capers, gherkins, parsley, Dijon mustard, and lemon juice in a bowl.