The mechanical operation of the backstay is fairly straightforward. But to get a better grasp of adjustment, we need to put aside the idea that the backstay simply pulls the mast backwards. This isn't entirely untrue, but in reality the stress is mainly directed towards the mast foot.
Imagine a batten with one end resting on the ground, and pressing down on the top bends the batten. That's more or less what happens to the mast when the backstay is lifted, giving the spar some deflection.
Assuming that your mast is slightly pre-stressed (the maximum resting bend corresponds to half the mast profile), the compression generated by the backstay tension will increase the forward bend in the middle of the mast.
The backstay can flatten the mainsail
As the mid-mast moves forward, the mainsail luff moves away from the leech, reducing the sail's hollow. The clew is moved away from the tack, flattening the bottom of the sail.
The backstay acts in the same way, but over a wider area, since to some extent the whole mast moves forward. The softer the spar, the easier it is to flatten the mainsail with the backstay.

On boats without backstays (small sailboats, multihulls), tensioning the mainsheet has the same effect. Mainsheet tension tightens the mainsail leech. The leech acts like a backstay, inducing compression and thus bending of the mast.
The vang can also help bend the mast. Tightening the vang lowers the boom and pushes it forward, bending the bottom of the mast.

Patatras can open the mainsail chute
When you compress the mast with the backstay, the head of the mainsail moves closer to the clew. This opens the mainsail leech, creating a twist.
Normally, every time you tension the backstay, you'll have to tuck in the mainsail (and/or vang) to keep the twist the same. Unless, of course, you're trying to get the wind's power out of the sail with more twist.
Finally, bending the mast also reduces its overall height from the spar, which will slacken the mainsheet. You may see horizontal creases indicating a lack of tension.
To compensate, and bring the mainsail trough as close to the mast as possible, you need to tension the luff with the halyard or cunningham. Conversely, when easing the backstay, remember to release the luff tension and ease the sheet or vang.
Have fun studying the effects of backstay trimming in different wind conditions. By mastering this adjustment, you'll have an additional, highly effective tool for controlling the shape of the mainsail.