If you are looking for a good riverside in London to relax with the family, then I will suggest the following places:
Southbank
Ok not so relaxing as it is in the middle of the city, but the views are great and there are restaurants and art galleries and a carnival atmosphere most days, so a great place to take the kids I would say!
Camden TownNot a river, but a canal goes through there. It can get very busy and touristy, but it is also a very entertaining place.
Hammersmith BridgeThis area has some nice restaurants and cafes by the river you can enjoy.
Chelsea embankmentNice river, quiet and leafy part of town.
To be honest though, if you want somewhere quiet and by a large body of water, then why not go to Hampstead Heath or Hyde park on a sunny day... They have "ponds" rather than rivers... But it is very beautiful!