I think the rule is: the number of triangles indicates how many edges you have to draw along.
Ignore the line I drew.
This occurred near the end of the music challenge section (in the maze like room). The puzzles are random there (but always solvable as far as I know), so I cannot get it back, but I am still curious.
Thanks