SVD is a dimension reduction technique. I will explain you this with an example.
Suppose I have a document with 1000 thousand words in it. There are 200 unique words in the document.
In text mining we represent the document by frequency of these words.
It may be also possible to represent the document with less than 200 words, for doing this we perform SVD on the document.
I hope now SVD make some sense to you.
For example if "Barack Obama" were part of the 200 unique words, those 2 could be reduced to 1 (btw, you don't choose this, the SVD just realizes it). Slightly more generally, it could be the case that subjects tended to have groups of words in common -- it may be more "natural" (in the SVD-orthogonal coordinate system sense) to use that grouping of words to describe the text.