They host leaders Warrington Wolves on Friday sitting bottom of Super League following a five-match losing run and having won just once in 10 games this term.
Their latest reverse was a 37-20 defeat at Hull FC when the West Yorkshire club showed signs of recovery only to fall into their old ways.
Talking to The Yorkshire Post, McGillvary said: “In patches we’re decent - not good but decent - but then we just do silly things.
“There’s silly errors which aren’t helping. It’s not just one person it’s throughout the team and it’s putting the team on the back foot all the time.
“Players are down on confidence and generally we’re just struggling and not playing smart but we can’t keep feeling sorry for ourselves.
“We maybe need one win and that could get our season going but we are struggling at the moment big time.
“We just have train hard and work even harder.”
Huddersfield finished third last term but are unrecognizable from that team now.
McGillvary, 27, insisted: “We know we’re capable of beating any side in this league but we can’t think we can do that just by playing in patches.
“We need to put an 80 minutes together. Hopefully we’ll do that against Warrington.”
Despite his side’s wretched displays, he has continued his own prolific form from last season when he finished as Super League’s leading try-scorer.
McGillvary scored his 10th try of the year at Hull on Friday and has now crossed in seven consecutive matches,
But he maintained: “It means nothing. Absolutely nothing.
“I don’t feel happy. We’re at the bottom of the table and I’d rather have no tries and be in the top four. I don’t really care about that.”
One positive was the debut of fellow winger Gene Ormsby who showed up well having joined on a month’s loan from Warrington.
“He’s a good player and I’ve played against him a number of times,” added McGillavry.
“I’m happy for him and he did well - it was a strong debut.
“He cant play next week (against Warrington) but he should go well for us.”