1 Do you want an agent who does a lot of editing on your work before sending it out to publishers? Most agents will have worked at publishing companies before going over to the dark side. A former editor may be unable to resist the urge to edit your work which, depending on your point of view, may be a good thing. On the other hand, you might prefer the sort of agent who settles down happily with a stack of miniscule-print contracts and enjoys quibbling over percentage points, in which case a background in rights would be good.
2 Do you want to be a big fish in a little pond? In which case you want a small agency. Your agent's income will be directly linked to yours, so they've got extra reason to sell your books (and generally be nice to you!). A big agency may make you feel they're too busy dealing with their star authors to have time for you. The number of authors each agent represents is also relevant - see also 6.
3 But a small agency may not have as much clout as a large agency, nor are they likely to have the same range of experience ie they tend to specialise in one kind of writing. A larger agency however will have specialist departments in children's writing, television/film etc and may have offices around the world. (Smaller agencies will have agreements with other agencies to bring in the expertise when needed, so this shouldn't be a dealbreaker.)
4 Personality. Top of my wish list was the desire not to be frightened by my agent and some literary agents I have met are terrifying. But I'm easily scared. Some writer friends relish the formidable qualities of their agent. Others couldn't care less so long as the agent does their job.
5 Age. Young agents are enthusiastic and keen to make their mark, but they may not have either the contacts in publishing or the experience. On the other hand, they may be actively looking for clients. Older agents have shedloads of experience and contacts - and existing clients.
6 Money. No reputable agent should ask you for money up front. Full stop. As to the percentage, I'd prefer they took 15% and represented fewer authors. But you might be happier with 10%.
Think about what's going to suit you. Go to talks at literary festivals and conferences, ask other writers, join writing societies and groups and generally get out there and do some research before drawing up your 'hit list'. Only an idiot would go to all the expense and effort of getting a positive response from an agent and then go all wobbly at the thought of being represented by them. Don't be that idiot.