The ‘mathematical formula’ you mention gives the volume of the object but says nothing whatsoever about the shape, which is what you need.
I haven't used WorldEdit, but from the example formula and the wiki documentation, the formula is a boolean expression evaluated over every block in the relevant area. The question the formula is answering is, for every block at some position (x,y,z), is that block part of the object you want to create?
In order to use this you need the inequalities which define shapes. For example, a sphere of radius 5 has the inequality
r <= 5, but WorldEdit gives you x,y,z and not r so you must use the formula for Euclidean distance to compute the radius yourself:
sqrt(x^2+y^2+z^2) <= 5 or equivalently
x^2+y^2+z^2 <= 5^2.
The torus formula they give works by computing the xy plane distance from a circle
0.75-sqrt(x^2+y^2) and then squaring that and
z^2 to obtain the 3-dimensional distance from
that circle, then comparing that against the desired radius. (All of these can be thought about as applications of the Pythagorean Theorem — two coordinate axes are the adjacent and opposite sides of a right triangle, and distance is the hypotenuse — but the torus example is complicated by effectively using cylindrical coordinates.)
There's really too much to say about how to do this for a single question; you'll have to find your own source for mathematics information, but the relevant subject is algebraic geometry. You may also find material on constructive solid geometry (CSG) useful, but a lot of discussions of that focus on the boolean-operations aspect (which you may want for more complex shapes) rather than the inequalities defining individual shapes.
However, you did specifically ask about making triangles. A triangle is a two-dimensional shape, so I'll show you a prism (extruded triangle) instead. These sorts of shapes which have only flat sides can be defined in a simple way: think about taking an infinite volume of stone, slicing it along some plane, and throwing away half. Each such plane is a linear equation (rather, linear inequality for our purposes) and we can just write a logical and:
x > -5 && x < 5 && y > -5 && z > -5 && x + y <= 0
This will create an 10×10×10 object which looks like a right triangle in the yz plane. The first two parts give it a limited extent in the x direction (WorldEdit probably has some limit on this anyway), the next two are the straight sides of the triangle, and the fifth is the sloped side.
A general linear inequality looks like
a*x + b*y + c*z <= d
where a, b, c, and d are some numbers you choose. With this you can define any flat surface you want, but it'll take a small amount of study to understand how. Note that every condition in the above formula has this form, if you set some of a,b,c,d to zero; for example, the fifth condition has a=0, b=1, c=1, d=0.
Questions of the form ‘How do I write an inequality for this solid shape?’ may be on topic at Mathematics Stack Exchange, but I recommend you consult available references first.
One thing to note is that formulas for three-dimensional objects are often given as equations which define the surface of a shape, whereas you want an inequality defining the interior. But, given the surface it may be possible to derive the interior. For example, Wikipedia says:
An implicit equation in Cartesian coordinates for a torus radially symmetric about the z-axis is
This formula is identical to WorldEdit's example except that it has
= r^2 instead of
< r^2; just try doing similar alterations to equations you find. (If you get the direction of the inequality wrong, you will convert everything but your shape into stone; I hope WorldEdit has good undo.)