Parallelism is one of the central figures of speech. It is a figure of sound patterning, as well as being a figure of construction. Parallelism nearly always involves something that is heard - a figure of speech that appeals to the sense of hearing. There are many ornaments of language which we all know depend on some sort of patterning, or repetition, of sounds. Rhyme is only the most obvious of them.
Many figures of construction are connected with the order of words, and with the patterning (or making of patterns) of thought. After the figures of comparison, this is the most common way in which we use figures of speech to "ornament" our language. In rhetoric, for example, orators often use a particular form of repetition to emphasise their ideas - they repeat similar words in identical structures. This is parallelism; other writers expand the use of the word in different directions. "Correspondence, in sense or construction, of successive clauses or passages; an instance of this; a sentence or passage which exemplifies this." (OED).
Many other famous orators have used parallel structures. Shakespeare has King Henry V improve his troops' morale by repeating the structure ["we" + noun phrase] in "We few, we happy few, we band of brothers" (Henry V, IV iii 60). "Never in the field of human conflict was so much owed by so many to so few." (Winston Churchill, Aug 20 1940[]) has three occurrences of the structure ["so" + quantifier]. (A more special term for the kind of parallelism in both of these examples is anaphora.)
The special form of parallelism that balances two opposing ideas ("on the one hand" ... "on the other") is called antithesis. A form that balances two clauses or similar units, in terms of their structures and lengths, is the isocolon. Paradiastole was defined in the 1998 Journal of Modern History (1998; 70 150) as "redescription of one thing as another - I am brave, but you are reckless; I am frugal, but you are stingy" (cited in OED).
See also a note on spelling parallel.