“לֹא כֵן הָרְשָׁעִים כִּי אִם כַּמֹּץ אֲשֶׁר תִּדְּפֶנּוּ רוּחַ”

“Not so the wicked, but [they are] like chaff that the wind drives away.”

(The allegory continues) Unlike the righteous persons that are described as a solid tree close to it’s source of water,
The wicked is detached and unstable, like a chaff.  Chaff, is the waste of the fruit. While it is attached to the fruit and the seed it has value, once detached, the chaff is worthless, gone with the wind.