Hm. I've been to DragonCon now twice (as I'm local) and it is a far stronger place for art than books from what I've seen. I've never seen a place for book sales at DC other than mostly comic books/compiled webcomics being sold in the comic artist alley as well as within the art show space.
That said, I remember one fantasy writer sharing a table with an artists sitting near a few webcomickers. I also remember these guys in the art alley (not comic side but art show side): https://www.facebook.com/WinstonandBaum1 . I made a point to talk to the author since he's a self-publisher and ask him about sales. He had a modest number of books and sold them all I think (but we're talking a dozen or two dozen IIRC).
I also remember Janny Wurts in the art show artist alley and she had lovely fantasy prints and copies of her paperbacks as well.
However- I remember two different authors who approached promotion at DC in a different way.
1) One year I stood in thewalk-up ticket line for 3.5 hours. Rachel Aaron (who had yet to release her Eli Monpress series) was actually passing out hard copy excerpts of her first book while we were in line. I think most people took the copies (we were seriously bored and hot). For me, this sample was enough to convert me to becoming a follower of her blog and eventually buying her books.
2) Last year I was outside of the comic alley and people were passing out bookmarks with "free book codes"... remember taking them and a few others going "hey cool" when the person explained what it was.
Alternative also to bring to your attention. Usually the "Decatur Book Festival" runs the same exact weekend as DragonCon. I think there has been some coordination between the two events in past years, but the audiences are very VERY different. (IN Atlanta, most "normal people" run away from DragonCon with the exception of coming down to see the Saturday morning parade. However Decatur is for the progressives and for those who value independently owned businesses, bookshops, music stores, etc.)
I'm not sure how much/how difficult it is to actually do the other event but it is a short train ride away from the main downtown area and possibly a more interested audience: http://www.decaturbookfestival.com/Community/index.php
In any case, will see if I can drag one or two folks who have sold at DC to this thread to give advice.
PS - if you DO table, I still think it advisable to have a hard copy (one at least from each author)... people actually do respond better to seeing a book and then picking up a flyer, bookmark, or whatever.