It is indeed sort of an Icee in a tube, or a type of popsicle. It comes in a few brands, like Chūpet (チューペット), Chū-Chū Ice (チューチューアイス) and Pokkin Ice (ポッキンアイス). Once it's frozen, you can twist/snap the tube in half at the midpoint and squeeze out the frozen slush inside. Or you can just refrigerate one, snip off the tip of the tube at one end, and drink the contents like Cirno's done.
