I would say reset everything to default (no extensions, etc), and then re-add extensions one by one, until you find the culprit.
Or, on Chrome, go to: chrome://memory-redirect and post resulsts.
Are you running latest version?
So I did reset the browser and added back my extensions. I think its a little better but not much:
Google Chrome 31.0.1650.63
589,064k 13,539k 602,603k 751,848 k 436,308 k
Note: If other browsers (e.g. IE, Firefox, Safari) are running, I'll show their memory details here.
Processes
Memory Virtual memory
PID Name Private Shared Total Private Mapped
10520
Browser
176016k 15888 k 191904k 204760k 137868k
5804
GPU
64812k 5656k 70468k 95180k 28368k
15292
Extension
AdBlock
64956k 5760k 70716k 88588k 23604k
6580
Extension
Evernote Web Clipper
11168k 6084k 17252k 22828k 23604k
920
Extension
Motorola Connect
23780k 6780k 30560k 48388k 26096k
8604
Plug-in
VLC Web Plugin
15152k 6544k 21696k 3792k 22356k
12100
Extension
JewishMusic Stream
21360k 18740k 40100k 30492k 26172k
9736
Tab (Chrome)
About Memory
10636k 15304k 25940k 18884k 23604k
4716
Tab
Post reply
59664k 22988k 82652k 66104k 26096k
12428
Tab
New Tab
21736k 16072k 37808k 29488k 23608k
8876
Extension
Screen Capture (by Google)
8184k 12152k 20336k 16824k 23604k
5000
Tab
Gmail - Email from Google
56000k 21724k 77724k 63228k 25664k
13220
Tab
Bank of America — Banking, Credit Cards, Mortgages and Investing
55600k 22324k 77924k 63292k 25664k
Σ 589064k 765080k 751848k 436308k