I didn´t look on what exactly you were doing there. I just commented on the overall Idea. As I said, its hard to really Balance.
Basically, if you dont allow breaking of Wood for example, the whole idea is next to Pointless and will only Ruin worlds. If you allow Breaking of Harder materials, it will punish Players who build nice. Or just want to have a nice Hut.
You might make the Broken Blocks regenerate themselves, that would counter random Destruction. But give an Increased Difficulty.