PDA

View Full Version : Forum Slowness and other issues.



KnowTheCosmos
2013-Apr-19, 12:10 AM
Hi all,


We're currently recovering from an external attack on www.CosmoQuest.org, which has been causing many issues with our server. The problem has not been completely resolved, however a large amount of the IP addresses causing the attack have been blocked, as well as placing some stricter rules for access requests to the site. You should be seeing a quicker response from the site and the forum. As you can see, the buttons and images have been fixed, which was caused by an issue with a server change that had an error which was needed to corrected manually. We've also fixed the issue with Tapatalk and you should now be able to access the forum from your mobile devices.

Thank you all for your patience as we worked to get everything back up and running. I'll be taking down the announcement at the top of the forum about the DDOS attack in just a few moments.


Best,

Scott
(KnowTheCosmos)

madman
2013-Apr-19, 04:24 AM
my pics have disappeared from my albums


http://cosmoquest.org/forum/album.php?albumid=293
http://cosmoquest.org/forum/album.php?albumid=343
http://cosmoquest.org/forum/album.php?albumid=345
http://cosmoquest.org/forum/album.php?albumid=348

***********************

also, member "LookingSkyward"'s pics are gone too.
http://cosmoquest.org/forum/album.php?albumid=339

****************

apart from that all other member album (and board) pics seem to be available and displaying fine.

can our albums be reactivated or do we have to re-(up?)load our pics to our albums

madman
2013-Apr-19, 04:34 AM
test pic

http://cosmoquest.org/forum/album.php?albumid=352&attachmentid=18378

***********************

none of my old pics appear in the "upload storage space" but i can upload new pics.

also i'm using the "url" tag as the "img" tag isn't showing my test pic on this page.

KnowTheCosmos
2013-Apr-19, 05:49 AM
I've created a trouble ticket for your specific issue. Thanks for the head's up.


my pics have disappeared from my albums


http://cosmoquest.org/forum/album.php?albumid=293
http://cosmoquest.org/forum/album.php?albumid=343
http://cosmoquest.org/forum/album.php?albumid=345
http://cosmoquest.org/forum/album.php?albumid=348

***********************

also, member "LookingSkyward"'s pics are gone too.
http://cosmoquest.org/forum/album.php?albumid=339

****************

apart from that all other member album (and board) pics seem to be available and displaying fine.

can our albums be reactivated or do we have to re-(up?)load our pics to our albums

HenrikOlsen
2013-Apr-20, 06:41 PM
I've created a trouble ticket for your specific issue. Thanks for the head's up.
Just to dial back the criticism a tiny bit in favor of looking for solutions, part of your communications problems might be alleviated by giving the admins here read access to that ticketing system so they can have some idea what you're doing without having to beg for scraps of information like the rest of us.

This naturally presumes that you're actually using it as a communications tool rather than a dump for bug reports.

Noclevername
2013-Apr-23, 11:05 AM
Just to dial back the criticism a tiny bit in favor of looking for solutions, part of your communications problems might be alleviated by giving the admins here read access to that ticketing system so they can have some idea what you're doing

Seconded.

Jeff Root
2013-Apr-24, 02:07 PM
Was there actually a distributed denial of service attack?

If so, then who or what was it directed against?

What became of it?

Where did it come from?

How did you choose the IP addresses to block?

How does a computer user determine whether his computer
participated in the attack?

What identifying name has been given to the attack malware?


If there was no DDOS, then what led you to think there was?

Have you unblocked the blocked IP addresses?

If not, why not? Do you still consider them a threat?

-- Jeff, in Minneapolis

Arneb
2013-Apr-25, 01:21 PM
Jeff, I know we are told to avoid do-it-yourself moderation, but could you, just for comfort of us other users, tone down the FBI interrogator routine a bit?

Jeff Root
2013-Apr-26, 03:11 AM
That's just what we should be told without having to ask,
but I've learned that at a lot of places, you have to ask.

I did wait five and a half days before asking.

-- Jeff, in Minneapolis

HenrikOlsen
2013-Apr-27, 12:51 PM
There's also different ways of asking.

A third of your questions are likely impossible to answer, the rest indicates the admins are incompetent in your eyes.

Why would you expect to get questions framed that way answered?

Arneb
2013-Apr-27, 01:22 PM
Exactly.

Jeff Root
2013-Apr-27, 01:41 PM
A third of your questions are likely impossible to answer,
The only one likely to be unanswerable is the one asking
where the attack came from. But it may be answerable, too.

All the other questions should have definite answers.

Answering how a computer user can determine whether his
computer participated in the attack will likely require a link
to a third party.

-- Jeff, in Minneapolis

HenrikOlsen
2013-Apr-27, 01:46 PM
Yes, it's not quantum physics, every question has a definite answer.
Whether the information to find that answer is available is a very different thing.

Jeff Root
2013-Apr-27, 03:11 PM
With the *possible* exceptions of the questions about the
source of the attack and how to determine whether one's
computer was a vector in the attack, all the other questions
should have definite answers which do not require any
further research to post here.

If it is known that an attack took place, then the target of
the attack is known and can be stated.

Whether the attack is still in progress or has ended would
be known if an attack actually happened. That requires no
additional research. If it has ended, the reason it ended
may also be known and can be stated. That is pretty much
what I meant by "What became of it?"

There may or may not be any good intelligence on where
the attack came from. If there is, it can be stated now.
If there isn't, *that* can be stated now.

How the IP addresses were chosen to be blocked obviously
does not require any further research to answer. It can be
explained right now.

I don't know whether identification of specific computers as
vectors in the attack is important to the people overseeing
the Cosmoquest software, but if it *is*, then they probably
have a good answer to this question in hand already, and
can answer it now. Whether it is important to them or not,
they can link to some web page where the malware used in
the attack is described and the question answered. Certainly
they wanted to know about the malware themselves. They
must have read about it online. So they have a link and can
provide it right now.

If there was a distributed denial of service attack, it and / or
the malware which perpetrated it would have been given a
reference name. The people at Cosmoquest who declared that
a DDOS attack took place must know this name. What is it?

Something led the Cosmoquest people to think that a DDOS
attack was occurring. What was it? No research is needed
to answer that question immediately!

No research is needed to answer the question of whether the
blocked IP addresses have been unblocked. If they haven't,
no reasearch is needed to answer the question of why they
haven't. Both questions could be answered right now, with
knowledge already in the head.

If they consider the attack to still be a threat, they can say
so right now, without having to do any research.

This should be easy.

-- Jeff, in Minneapolis

.

Ara Pacis
2013-Apr-27, 03:57 PM
Even if there was really a DDOS attack specifically directed at cosmoquest (instead of targeting the host), why would they tell us details? Are you a CS expert trying to offer help?

If it was an attack specifically targeting cosmoquest, then the assumption would be that it was caused by someone who has a problem with the owners or members and someone who has expertise in DDOS attacks. Perhaps such as person is themselves a member. Perhaps they'd want this information to know how effective their attack was in order to plan another one in the future, or perhaps they want to know the details to figure out if they're likely to be caught and prosecuted. Does anyone here know anyone who fits that description?

Swift
2013-Apr-27, 06:41 PM
OK, enough meta-discussion about Jeff Root's questions.

I have no clue as to the answers, and it is neither my expertise, nor my interest to find the answers. If someone from the build team choses to answer, that's their call (I wouldn't recommend losing sleep waiting for the answers). I'm also not sure it would be a good idea to make such answers public anyway.

Arneb
2013-Apr-27, 09:16 PM
OK, enough meta-discussion about Jeff Root's questions.
Of course.


I have no clue as to the answers, and it is neither my expertise, nor my interest to find the answers. If someone from the build team choses to answer, that's their call (I wouldn't recommend losing sleep waiting for the answers). I'm also not sure it would be a good idea to make such answers public anyway.
The subject is certainly interesting. Could you help us and ask someone in Adminland if they could fill the membership in on the story? I think everyone would appreciate your (and their) effort.

Jeff Root
2013-Apr-28, 01:28 AM
If answering any question would compromise security or
complicate operations in any way, then of course that info
should be withheld for as long as required, and that fact
can be stated now. It isn't obvious that answering any
of the questions would raise such a problem.

We have been told that there was a DDOS attack on
www.CosmoQuest.org. That is alarming, unusual in the
extreme, and one of the most newsworthy events ever
to happen here. Of course I want to know more. I'm
on the edge of my seat.

If it was a mistake, and there was no DDOS attack, then
that, too, is alarming, extremely unusual, and highly
newsworthy. Either way, I want to know more.

-- Jeff, in Minneapolis

.

swampyankee
2013-Apr-28, 03:48 PM
I worked in software development for a couple of decades; DDOS attacks are difficult to deal with, and, due to botnets, a significant number of the computers that are involved have been corrupted. I presume that the site's admins are aware of the the comp.risks news group, archived at http://catless.ncl.ac.uk/Risks/, but some of the posters may not be. It is a great source of information about various security issues.

Among security "experts," there is a very strong tendency to hide information, on the assumption that "the bad guys shouldn't know." The flaw in this logic is, of course, that the bad guys have already found this information.

slang
2013-Apr-28, 09:30 PM
I worked in software development for a couple of decades; DDOS attacks are difficult to deal with [...]

They can be difficult to deal with. Sometimes it's not so difficult, depending on the type of attack, the size, and of course the resources the victim has available to deal with the problem.


Among security "experts," there is a very strong tendency to hide information, on the assumption that "the bad guys shouldn't know." The flaw in this logic is, of course, that the bad guys have already found this information.

Some have. Some haven't. Some bad guys are remarkably clever engineers (even if in spirit rather than title). Some are really, really dumb folks. Why help the latter? We will not publicly discuss matters that have to do with site security. Continuing to push for answers will just lead to closure of the thread.

Personally, I'm thankful that for once the build team was quick to inform us (members) of the reason, by their analysis, that we might be encountering slow performance, and that it was being dealt with.

HenrikOlsen
2013-Apr-28, 10:00 PM
They can be difficult to deal with. Sometimes it's not so difficult, depending on the type of attack, the size, and of course the resources the victim has available to deal with the problem.



Some have. Some haven't. Some bad guys are remarkably clever engineers (even if in spirit rather than title). Some are really, really dumb folks. Why help the latter? We will not publicly discuss matters that have to do with site security. Continuing to push for answers will just lead to closure of the thread.

Personally, I'm thankful that for once the build team was quick to inform us (members) of the reason, by their analysis, that we might be encountering slow performance, and that it was being dealt with.
Quick? Based on timing, it took escalating the problems to something potentially affecting funding of the main site to get any kind of response from them and after getting it, it's been crickets again.

I don't actually give a damn about whether the build team understands how frustrating not getting any information is, I do however give a damn that even though they claim to understand, nothing's changed in that regard, they haven't addressed how to fix the problems of communication.

And I fully understand the frustration of the forum admins who apparently get about as little information as we do and still has to stand in for the ones with actual power.

slang
2013-Apr-28, 10:13 PM
Quick? Based on timing, it took escalating the problems to something potentially affecting funding of the main site to get any kind of response from them and after getting it, it's been crickets again.

The escalated issue you are referring to was a different one, that was the missing icons and images, wasn't it? Yes, communication was terrible as usual in that case. Don't get me started on testing.

HenrikOlsen
2013-Apr-28, 10:29 PM
The escalated issue you are referring to was a different one, that was the missing icons and images, wasn't it? Yes, communication was terrible as usual in that case. Don't get me started on testing.
Somewhat related though, since the images "fix" was an attempt at managing the slowdown that was later shown to be caused by the DDoS.

slang
2013-Apr-28, 10:53 PM
Somewhat related though, since the images "fix" was an attempt at managing the slowdown

True.


that was later shown to be caused by the DDoS.

It was? I assumed from KtC's info that it was a transient thing (as usual in such cases). But maybe I misread or missed something.

HenrikOlsen
2013-Apr-29, 12:10 AM
I read KtC's info as meaning that the slowdowns experienced were caused by the DDoS and the "fix" was made before the root cause was realized, with the DDoS being detected upon further investigation.

Incidentally, this is something that we quite definitely could get clarification about without any effect on security.

Jeff Root
2013-Apr-29, 04:16 AM
I don't believe that any of the answers to any of my questions
pose the slightest security risk to Cosmoquest, or would be of
the slightest value to any hacker. The answer to every question
I asked is either already known to the attacker, or is of no use
to him, or both.

There is no good reason we shouldn't have the answers to my
questions above immediately. If you think there is any good
reason at all, please say what it is and why you think it is
relevant.

A distributed denial of service attack is an extremely unusual
and alarming event. It is vandalism. I want to know who was
attacked. Was it Cosmoquest? Or someone else? I want to
know who did it. I want to know if the attack is still going on.
I want to know if my computer has been taken over by the
vandals as a vector in the vandalism. I want to know whether
my personal stuff has been vandalized. I want to know why
you want to keep the answers to these questions secret. And
if you won't tell me, I want to know why you won't tell me.

-- Jeff, in Minneapolis

slang
2013-Apr-29, 09:19 AM
I don't believe that any of the answers to any of my questions
pose the slightest security risk to Cosmoquest, or would be of
the slightest value to any hacker. The answer to every question
I asked is either already known to the attacker, or is of no use
to him, or both.

There is no good reason we shouldn't have the answers to my
questions above immediately. If you think there is any good
reason at all, please say what it is and why you think it is
relevant.

And once again you forget that this person is not the only one messing about. Exposing methods would not affect him (or her). But it would help anyone else who might think to try something similar, and how to implement it in such a way that those things done this time will not work the next time. Your assertion that DDoS attacks are extremely unusual is wrong. It is rare for some simple forum to be the target, but hosting companies are often under attack, as are many other types of businesses. I tell you this as an IT engineer whose employer had to hire a third party to provide additional DDoS protection at the ISP level.

You cannot tell from the signs in logfiles who is behind an attack. You can't know who or what it was directed against (was it against Amazon hosting?), except that it hits your webserver, intentional or not. It looks the same if it was some misconfigured advertising botnet draining resources (and thus denying service).

You will not get your answers because we don't want to help anyone else. And with that, this thread is closed. Which is unfortunate, because Henrik's last question was a good one.