It depends on whether you’re replicating the conditions for actual calligraphy – the type of brush used, orientation of the board and texture of the board will all affect how realistic your practice is. The motions you apply for moving the brush and preparing it for writing are also a part of the skillset. If you can replicate these things with water and a Buddha board, then yes, such practice may be of use.
Details on traditional calligraphy (I’m speaking for the Chinese tradition: I don’t know if Japanese calligraphy has particularities that don’t exist in Chinese calligraphy, kana and the way the brush is held aside, but what I’ve seen so far seems similar as far as materials go. Technique may differ between styles.)
Real traditional brush calligraphy is done on rice paper (which is slightly crinkly and very absorbent) with black ink and brushes composed of a bamboo handle and animal hair (like horse or wolf hair, if my memory serves me). The ink is usually made by mixing an ink stick with water in an ink stone with a grinding motion. It sometimes comes premixed, in which case it can be poured directly into the ink stone or into a small, shallow dish. Before writing, the head of the brush should be lightly pressed against the dish or ink stone with a stroking motion in order to remove excess ink.
Practically speaking, you’ll want to make sure your Buddha board is flat on the table and that your brush is a calligraphy brush. Ordinary paintbrushes will not do, especially those made with synthetic fibres. Make sure that your water is in a shallow dish (something the size of sushi soy sauce dipping dish will be perfect). Dip your brush into the water and then stroke it against the edge of the dish in order to remove excess water. Rotate the brush as you do so. The objective is to shape the head of the brush into something resembling a flame or flower bud: pointed at the tip, smooth and round where it meets the brush. Thereafter, you’ll have to write on the Buddha board with the correct motions in order to create all the contours of the various strokes. You’ll probably find some information if you look up Japanese or Chinese calligraphy.
The reason I’m fairly certain that practising with water on a board that darkens where it is wet is helpful is because it’s one of the methods used for training in Chinese calligraphy: I have a few booklets of calligraphy exercises made using such paper with kanji templates printed on them for the student to trace. I used them when I attended calligraphy classes as a child. (This also means that the Buddha board is unlikely to be based on any sort of traditional Zen practice since it’s really just a very practical calligraphy exercise. I’m just putting this out there for those tempted to buy into this out of admiration for Zen Buddhism, since I saw Zen being mentioned in an ad for a Buddha board – I’m not sure if it’s ‘authentic Zen’, so to speak, even if it’s good calligraphy practice.)
Ultimately, however, developing skill in calligraphy (shōdō) requires more than mechanical practice: you need to learn what makes a character beautiful. The most common way of improving one’s calligraphy is imitating the work of masters while attempting to replicate the angles used and the thickness of the strokes. However, I’ve found it more helpful to learn general principles that contribute to the beauty of a kanji. This can be done through observation (e.g. noting character proportions, stroke density and the general disposition of various components) or through reading works on calligraphic principles. I know that such texts exist in Chinese, and others surely exist in Japanese, but I don’t know if any of them have been translated into English.
In any case, I wish you all the best with your practice, and I hope that you enjoy it even if you don’t get immediate results.
(PS: if you’re wondering why you should trust me… I picked up a course from China on fountain pen calligraphy two years ago and spent about three months on it. I do my best to apply the aesthetic principles I learnt to brush calligraphy as well. As for what the results might look like… my profile picture is something I wrote.)
To add on to this as someone who started learning Chinese as a toddler (semi-native speaker, if you like, since I admitted usually spoke English at home, while watching TV almost exclusively in Chinese), I see kanji writing themselves in my head when I hear or think about them, and that gives me another way to retain them. More practically, learning to write complex kanji helps you to remember details like additional long strokes or dots that might help to differentiate them from other similar kanji. It may seem like a chore to learn the correct stroke order now, but you’ll find that there’s often a certain logic to them, and that as you go along, you’ll get faster and faster. It’s really the same way we learnt the English alphabet: write a letter, say its name; write a kanji, read it aloud. Plus, if you ever need to write kanji by hand, your writing will be much more legible if you follow the correct stroke order. The general idea is this: kanji are written component by component, top to bottom, left to right. There are some exceptions, but if you consider the overall stroke order for each character, you’ll find that this general flow applies to all kanji.