Brazil is in South America, also known as Latin America... so - yes, they are latino - but I understand your point. Some Argentines believe they are European, but believing something, doesn't necessarily make it so.
Latin America generally refers to Central rather than South America, but anyway the point is that Latino as a category very explicitly excludes Brazilians, and refers to Spanish speaking peoples (although many Latinos don't actually speak Spanish).
What do we speak, then? I know here in the USA, many of us speak "SpanGlish"... part Spanish - part English. LOL - Especially in mixed cultered states, such as Fl., Ca., Tx., Az., etc.