The answer is in the context. If you use an unusual word, the context should make it clear what that word means. I used quite a few Italian words in A Single to Rome:
'Ecco, Carciofi all Guidia.' Teresa placed a steaming dish on the centre of the table, a tumbled mass of golden brown artichokes. Natalie inhaled deeply; the 'benvenuto a Roma' evening for Guy smelled delicious. 'Nothing but Roman specialities. Eat, eat,' Teresa said, ladling out the artichokes onto a plate and passing it to Guy, the guest of honour. She prepared a plate for Natalie. 'This is a traditional dish of the Ghetto,' she said, passing the plate to her.
Hopefully the reader has worked out that Carciofi are artichokes in Italian. All Guidia is Jewish style, which hopefully the reader guesses from the reference to the Ghetto. Benvenuto a Roma is explained earlier in the story, but I suspect most people could work it out from the context, especially as benvenuto is similar to the French for welcome (bienvenue). The next excample is from Another Woman's Husband (NB Lily is Martin's daughter).
Martin rolled towards her, the contours of his face catching escaped light from the street lamp outside. He looked young, his face blurred with sleep and, in the planes and angles of his bone structure, Becca caught sight of Lily underneath, like a palimpsest of the two people who meant the most to her.
A palimpsest is a manuscript which has been written on several times. I accept you wouldn't get that if you didn't know it, but I hope the idea of layers is present. So long as the context makes the meaning clear, there shouldn't be the hiccup in understanding that draws the reader out of the story. Therefore, you need neither dumb down nor baffle the reader but write exactly what you want to write.