As mentioned, in many cases the customization of algorithms is the only way to achieve the target, but sometimes, some tricks can help to improve the learning even without changes of learning strategy!
Consider for example our XOR problem solved through neural networks.
Let's see how we can reduce considerably the epochs required to train the net.
read more here: http://textanddatamining.blogspot.com/
Here some results:
Original configuration (convergence after 600 cycles)
Net with recall neuron: convergence after 150 cycles.
The surface error for the most important synapses: (notice how the error slumps quickly)