# Intermittent site hang lockup?



## RAD (Aug 5, 2002)

For the past couple weeks I've had an issue where the site locks up. This happens at the same time from a PC trying IE and Chrome browser and an iPhone and iPad using either the IOS app or Safari browser. During the time I have the lockup I'm able to ping the site fine, no dropped packets and <20ms times and no problem accessing other internet sites. Latest lockup occured on 6/28 from about 14:25 to 14:22CDT.


----------



## peds48 (Jan 11, 2008)

yep, same here on MAC, iPhone and iPad


----------



## klang (Oct 14, 2003)

I've noticed it the last couple mornings.


----------



## MysteryMan (May 17, 2010)

I've had issues too using both Internet Explorer 10 and Google Chrome.


----------



## itzme (Jan 17, 2008)

Same on PC with Chrome and ipad.


----------



## scoop8 (Jul 18, 2009)

I haven't seen it yet but I only sign into the site twice daily, early morning & mid-late afternoon. I'm using IE9 .


----------



## Drucifer (Feb 12, 2009)

Yep, haven't notice it today so far, but for last few days, I have had connecting issues from my PC FireFox v21.0


----------



## klang (Oct 14, 2003)

Did it me again just now trying to refresh the list of forums. Firefox on Windows.


----------



## P Smith (Jul 25, 2002)

Devices,OSes, browsers, versions are irrelevant; the problem is with the site or its domain. Other places like AVSforum are OK.


----------



## Stewart Vernon (Jan 7, 2005)

Funny... I couldn't get to the site at all at one point yesterday... and just now, I experienced a delay in trying to get to this very thread!

Similarly... no problems getting to other Web sites when this is happening here. I am running Firefox beta 23 on a Mac.


----------



## inkahauts (Nov 13, 2006)

Add me to this list. At first I thought maybe it was a DNs issue on my end it something and rebooted my router and reinstalled my iOS apps and such. But then it still happened on my pic last night and my iOS apps today a couple times. Something weird is going on.


----------



## Drucifer (Feb 12, 2009)

Drucifer said:


> Yep, *haven't notice it today* so far, but for last few days, I have had connecting issues from my PC FireFox v21.0


Still here today.


----------



## David Bott (Jul 12, 2005)

Hi All...

Strange, as I personally have not seen the issue nor can I see anything server side. 

I have asked our host provider to look into it and see if he can see anything. As soon as I have anything to report, I will. 

Thanks


----------



## RAD (Aug 5, 2002)

Still happening on Saturday AM about 9:20CDT. If it helps, my ISP is Time Warner.


----------



## jimmie57 (Jun 26, 2010)

RAD said:


> Still happening on Saturday AM about 9:20CDT. If it helps, my ISP is Time Warner.


+1 at 9:32 am. ISP is Comcast / Xfinity


----------



## David Bott (Jul 12, 2005)

Ok, well I did find the in the mysql process log. So I am going to turn off email logs (the thing that leasts me check on email issues) as that is what that process is in the screen shot. Not sure this will do it, but it is something to at least try. Let me know if it improves.

EXIM is the mail server system.

(But while I am hopeful that this is the issue, I am not thinking it is.) Next move will be to upgrade MySQL and I will need to take the site down for a shot time to to that. (And not do it when we are traveling... Wife is driving why I play with this... http://www.outsideourbubble.com/live-coach-cam but want a stable connection for harder things like upgrades of the OS. )


----------



## damondlt (Feb 27, 2006)

Same issue here , and I have IE10


----------



## David Bott (Jul 12, 2005)

See what happens when you trying to make what some simple changes should be. The server goes down and MySQL, Apache and PHP fails to restart.

At this point, I am not sure what state the server is actually in other than currently slow. This could be because things need to cache.

I am going to give it some time now to settle and see what happens.

I want to thank a member that came in to help from the IP.Board community.


----------



## David MacLeod (Jan 29, 2008)

glad to help 
I am all over the place


----------



## gov (Jan 11, 2013)

Anyone try rubbing cheetah blood on the server ??


----------



## David MacLeod (Jan 29, 2008)

yes, all it did was lube the drives up.


----------



## David Bott (Jul 12, 2005)

Ok, well I was not sure you wanted me to use your name...SO...GOOD...THANKS DAVID for all your help this afternoon.

What you all do not know is that when I tried to upgrade MySQL, well, it all went very bad. I put our a call for help over on the IP.Boards site for admin help and look who was their...Our very own David. 

So thanks again. (And yes, I put out a note to look at replacing the server itself with a more current OS. It is not the server that is our issue, it is an older OS that has no upgrade path from 5.9 to 6. So we need to start fresh and move everything over to a new box even though this is a 8 processor machine with 16 gigs RAM etc.)


----------



## David MacLeod (Jan 29, 2008)

due to time issues on my end I cannot devote the time really needed but I am suspecting there is a socket issue with xcache and centos 5.9.
I could be wrong though, have not had chance to dig deeply


----------



## David MacLeod (Jan 29, 2008)

lets see if this works.


----------



## Stewart Vernon (Jan 7, 2005)

That answers some of the "what blew up" questions I had... I noticed the error page for a while today, some kind of SQL database error... then the Web site stopped responding even with an error for a while after that.... I decided to try again and see if we were back up yet.


----------



## David MacLeod (Jan 29, 2008)

some of it was also me restarting services after I started to work on it, so between 7 and 730pm mostly it was me.


----------



## David MacLeod (Jan 29, 2008)

seems ok now, I have been hitting links for 20+ min straight.
750 to 850 online when doing it and was smooth for me.
will monitor tomorrow.


----------



## David Bott (Jul 12, 2005)

Ok, we seemed to have some issues with the MySQL config. I THINK I have it sorted out as of the writing of this post. We shall see.


----------



## Stewart Vernon (Jan 7, 2005)

I noticed the site was down again earlier... forgot to notice the time... then I got distracted with TV and didn't check back in until now.


----------



## David MacLeod (Jan 29, 2008)

my dsl crapped out last night so was not able to keep eye on much, sorry


----------



## longrider (Apr 21, 2007)

Just experienced the issue today, 9:28AM MT, took about 2 minutes to get a response


----------



## RAD (Aug 5, 2002)

Also happened here around the same time.


----------



## David Bott (Jul 12, 2005)

Pondering, their is nothing in the logs at that time showing any issues with the database or connections etc. As far as 2 mins to get a response, that I would think the browser would have timed out before that. So even more lost.

When this happens...can you traceroute to the site? Wondering if it is a network issue and I can only tell that with a trace.


----------



## David MacLeod (Jan 29, 2008)

wonder if thats apache prefork (non threaded) bog
any idea how many online at that point?
server is also using suphp, there is evidence that can cause slowdowns too. unsure of who else is hosted on the server so could not really say if thats needed or not.


----------



## longrider (Apr 21, 2007)

David Bott said:


> Pondering, their is nothing in the logs at that time showing any issues with the database or connections etc. As far as 2 mins to get a response, that I would think the browser would have timed out before that. So even more lost.
> 
> When this happens...can you traceroute to the site? Wondering if it is a network issue and I can only tell that with a trace.


David, I will do a trace if it happens to me again. I know it is not my connection, I have verified that is functional while DBSTalk is not responding. As for the timeout, when I know it is getting close to timing out I usually stop that load and either try again or try a different page on the site. The second attempts or alternate pages also do not load, if I do let it time out and use the browsers 'try again' it times out very quickly the second time


----------



## David Bott (Jul 12, 2005)

Hi, thanks. 

What I am looking to see when you are seeing the issue is a traceroute. I ask for while your connection to other sites might be good, I am trying to see if it at the network center I am in. Thus as trace can show me is their are any issues network wise at the time.

I just made a few change, but I do not think it will de another other than to give me something else to look at time wise.

So, when it happens, note the time (The severs is in CT) and if you can catch a trace route while it is happening....great! Post it here.

Thank you for the help.


----------



## RAD (Aug 5, 2002)

I'll also try to get a traceroute next time it happens. I did mention that I'm able to ping the site while it's hung, <20ms times. I've also pinged with a 1400 byte packet and while response time a tad higher also works fine, no drops.


----------



## RAD (Aug 5, 2002)

Hung up again, both IOS client, IOS Safari broswer and PC with IE 9.0 browser happened around 13:00 CDT. While I was posting this the site hung up again for about a minute. Pings and traceroute taken while site hung below:

Ping statistics for 72.9.159.119:
Packets: Sent = 34, Received = 34, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 17ms, Maximum = 30ms, Average = 19ms

Tracing route to www.dbstalk.com [72.9.159.119]
over a maximum of 30 hops:
1 <1 ms 1 ms <1 ms 192.168.10.1
2 * * * Request timed out.
3 17 ms 9 ms 9 ms tge7-3.austtxs-er01.texas.rr.com [66.68.1.105]
4 9 ms 10 ms 7 ms tge8-1.trswtx1202h.texas.rr.com [24.175.41.193]
5 15 ms 15 ms 15 ms te0-10-0-0.austtxrdcsc-cr02.texas.rr.com [24.27.
13.96]
6 18 ms 23 ms 15 ms agg22.hstntxl3-cr01.texas.rr.com [24.175.41.48]
7 17 ms 19 ms 19 ms ae-4-0.cr0.hou30.tbone.rr.com [66.109.6.54]
8 20 ms 20 ms 18 ms ae-0-0.pr0.dfw10.tbone.rr.com [66.109.6.181]
9 17 ms 17 ms 17 ms 107.14.16.186
10 31 ms 18 ms 20 ms ae-2-70.edge3.Dallas1.Level3.net [4.69.145.72]
11 20 ms 21 ms 23 ms COLO4-DALLA.edge3.Dallas1.Level3.net [8.9.232.74
]
12 28 ms 19 ms 21 ms 72.249.128.110
13 22 ms 18 ms 25 ms dbnet.bott.net [72.9.159.119]
Trace complete.


----------



## David MacLeod (Jan 29, 2008)

when it bogs it does recover on its own now correct?
no sql or page not found errors now?


----------



## Drucifer (Feb 12, 2009)

Was doing fine until I hit the link in the email notification for this thread and got another delay of several minutes.


----------



## RAD (Aug 5, 2002)

David MacLeod said:


> when it bogs it does recover on its own now correct?
> no sql or page not found errors now?


Correct, at least since things were missed up yesterday afternoon/evening. It just sits there no timeout and eventually responds. There are a few times where I'm catching it come back after a lockup, it works for a minute or so then hangs up again and again comes back on its own.


----------



## longrider (Apr 21, 2007)

I had done this earlier for comparison, this is while the site is working fine:

Tracing route to www.dbstalk.com [72.9.159.119]
over a maximum of 30 hops:

1 4 ms 4 ms 4 ms 192.168.1.1
2 86 ms 49 ms 34 ms coloradosprings-pr732.kellin.net [198.57.1.1]
3 93 ms 51 ms 39 ms cls2-edge-03.inet.qwest.net [207.109.162.165]
4 83 ms 99 ms 89 ms dvr-brdr-02.inet.qwest.net [67.14.24.114]
5 102 ms 119 ms 69 ms 63.146.26.134
6 105 ms 101 ms 98 ms vlan51.ebr1.Denver1.Level3.net [4.69.147.94]
7 87 ms 347 ms 91 ms ae-2-2.ebr2.Dallas1.Level3.net [4.69.132.106]
8 87 ms 104 ms 95 ms ae-72-72.csw2.Dallas1.Level3.net [4.69.151.141]

9 83 ms 121 ms 99 ms ae-2-70.edge3.Dallas1.Level3.net [4.69.145.72]
10 93 ms 109 ms 109 ms COLO4-DALLA.edge3.Dallas1.Level3.net [8.9.232.74
]
11 102 ms 91 ms 97 ms 72.249.128.110
12 109 ms 63 ms 59 ms dbnet.bott.net [72.9.159.119]

Trace complete.

When the hangups happen I do not get any errors, just no response. I was not online at the time RAD experienced the issue


----------



## David Bott (Jul 12, 2005)

Well every long query error looks to be caused by TapaTalk...

/usr/sbin/mysqld, Version: 5.5.31-cll (MySQL Community Server (GPL)). started with:
Tcp port: 0 Unix socket: (null)
Time Id Command Argument
# Time: 130630 12:36:39
# [email protected]: dbstalk_ipb[dbstalk_ipb] @ localhost []
# Query_time: 52.141685 Lock_time: 0.000077 Rows_sent: 222 Rows_examined: 21973116
use dbstalk_ipb;
SET timestamp=1372613799;
SELECT p.*,m.member_id as author_id FROM tapatalk_push_data p LEFT JOIN members m ON ( p.author = m.members_display_name ) WHERE p.user_id=454547 ORDER BY p.create_time DESC;
# Time: 130630 12:50:51
# [email protected]: dbstalk_ipb[dbstalk_ipb] @ localhost []
# Query_time: 43.735035 Lock_time: 0.000088 Rows_sent: 171 Rows_examined: 16925238
SET timestamp=1372614651;
SELECT p.*,m.member_id as author_id FROM tapatalk_push_data p LEFT JOIN members m ON ( p.author = m.members_display_name ) WHERE p.user_id=442206 ORDER BY p.create_time DESC;
# Time: 130630 13:01:39
# [email protected]: dbstalk_ipb[dbstalk_ipb] @ localhost []
# Query_time: 345.505372 Lock_time: 0.000090 Rows_sent: 1451 Rows_examined: 143617078
SET timestamp=1372615299;
SELECT p.*,m.member_id as author_id FROM tapatalk_push_data p LEFT JOIN members m ON ( p.author = m.members_display_name ) WHERE p.user_id=417495 ORDER BY p.create_time DESC;
# Time: 130630 13:15:21
# [email protected]: dbstalk_ipb[dbstalk_ipb] @ localhost []
# Query_time: 7.772315 Lock_time: 0.000061 Rows_sent: 30 Rows_examined: 2969340
SET timestamp=1372616121;
SELECT p.*,m.member_id as author_id FROM tapatalk_push_data p LEFT JOIN members m ON ( p.author = m.members_display_name ) WHERE p.user_id=454231 ORDER BY p.create_time DESC;
# Time: 130630 13:30:03
# [email protected]: dbstalk_ipb[dbstalk_ipb] @ localhost []
# Query_time: 5.508243 Lock_time: 0.000045 Rows_sent: 21 Rows_examined: 2078538
SET timestamp=1372617003;
SELECT p.*,m.member_id as author_id FROM tapatalk_push_data p LEFT JOIN members m ON ( p.author = m.members_display_name ) WHERE p.user_id=440109 ORDER BY p.create_time DESC;

...I will need to send them a note as this could be the issue making for the locks based on the time frame you gave me.


----------



## David MacLeod (Jan 29, 2008)

tapatalk pushes to each member whenever watched item replied to ?


----------



## klang (Oct 14, 2003)

Wow, 143 million rows examined! My database experience was mainframe DB2 and now out of date but perhaps an index is needed?


----------



## David Bott (Jul 12, 2005)

klang said:


> Wow, 143 million rows examined! My database experience was mainframe DB2 and now out of date but perhaps an index is needed?


I do not disagree. But I do not know what tables or rows to index.  I would like them to give me some guidance on what might be needed or look into the issue.

It might not be the issue we are having...but it sure seems that the times match up and those are the only queries showing up in the slow query log.


----------



## MysteryMan (May 17, 2010)

Site became unresponsive at 07:41 Eastern Time this morning.


----------



## David MacLeod (Jan 29, 2008)

David Bott said:


> I do not disagree. But I do not know what tables or rows to index.  I would like them to give me some guidance on what might be needed or look into the issue.
> 
> It might not be the issue we are having...but it sure seems that the times match up and those are the only queries showing up in the slow query log.


an interim test would be to disable tapatalk push notifications for a bit and watch the slow log for a bit.
now its been up for over 24hrs slqltuner may show some useful info too since this is basically a default mycnf


----------



## David Bott (Jul 12, 2005)

Just installed an update from Tapatalk that they sent me this morning.

I am also working on installing the Sphinx for the search engine. So mySql will be going down for a few mins so access will be not possible.


----------



## David Bott (Jul 12, 2005)

Ok, had some issue with Sphinx as it is not as cut and dry as I thought. So right now I have the Tapatalk update/fix installed. So we will start their. But I still want Sphinx a some point.


----------



## David MacLeod (Jan 29, 2008)

chmod issue on wherever you stored the sphinx deltas/logs?
I'll be reinstalling that later today, last time only took 20 minutes or so.
this is still pretty pertinent, may help
http://www.invisionpower.com/support/guides/_/maintenance-and-server-configurations/searching-r21


----------



## David Bott (Jul 12, 2005)

I removed it all at this point to start over when I get the chance.


----------



## David MacLeod (Jan 29, 2008)

hopefully I'll be doing mine in about 2 hours, I'll document it and send the info when I do.
usually any issues are just pathings and its a pita to remember them sometimes.


----------



## David Bott (Jul 12, 2005)

Is anyone still seeing the issue?


----------



## longrider (Apr 21, 2007)

I have not experienced it so far today, but I was only online a little early this morning and then right now


----------



## RAD (Aug 5, 2002)

Working better on my iPhone and browsers. 

But the app on my iPad not so much. When I open it, the notifications panel opens then sits there sitting on loading feeds and that's it. I've tried uninstalling/ reinstalling the app, didn't help.


----------



## David MacLeod (Jan 29, 2008)

David Bott said:


> I removed it all at this point to start over when I get the chance.





David MacLeod said:


> hopefully I'll be doing mine in about 2 hours, I'll document it and send the info when I do.
> usually any issues are just pathings and its a pita to remember them sometimes.


took me about 15, let me know when you want to try it (tomorrow) and we can walk through sphinx setup.
tested with yesterdays tapatalk too and no issues, every now and then a tapatalk release gets through that does not work with sphinx.


----------



## David Bott (Jul 12, 2005)

RAD...I am not sure it is related to what we are trying to solve. But seeing I did just apply an update given to me from Tapatalk, the issue could be related to what you are seeing. But the issue really seemed to be PUSH notice related. So thinking it is not. (Did that come out right?)


----------



## RAD (Aug 5, 2002)

David, if it helps, the iPad issue started later yesterday not this AM. Sometimes I can get to another screen like current but then if it try to do something, like reply to this thread it hangs up.


----------



## David Bott (Jul 12, 2005)

Ok, so not related to the update. Good. But sorry you are having and issue.


----------



## David Bott (Jul 12, 2005)

Just and update...

Ok, now running the TapaTalk fix (Which is what I think the overall issue was.) from 11:13 AM ET this morning. I have not seen anyone say things were still bad...so I think that was it.

However...What we did do was...

Upgraded MySQL to ver 5.5

Also have XCache running again. (Had stopped it just in case) XCache speeds up php code by caching the php scripts in a compiled format vs needing to compile it with each call. The entire site runs off PHP and MySQL

Also now running the Sphinx search engine. This is a very fast search solution vs MySQL standard search. Each morning at 4:00AM CT it will slow the site down as it reindexes for just a few mins. (Side note...new posts will be added to the search indexes within 10 mins of posting vs real time.)

So....Now let's see how this all goes.


----------



## David MacLeod (Jan 29, 2008)

would be curious to see what browsers in use if anyone has issue too.
some people, we won't mention them by name so as to not smoke them out, still use IE8.


----------



## Stewart Vernon (Jan 7, 2005)

I haven't seen any problems so far today (knock on wood) and I have been on a few times, and actually logged in and wandering a couple of times.


----------



## peds48 (Jan 11, 2008)

Do not know if it has anything to do with this, but iPhone app does not load


----------



## Dude111 (Aug 6, 2010)

I have been hsving this happen on a VBB site im on... Every now and then i cant get to any of his sites UNLESS I OPEN "WINIPCFG" and click RENEW ALL!! (Started after he updated his server,i have NO IDEA why its doing it.. I sent him a PM about it)

I wonder if that same thing is happening here?


----------



## MysteryMan (May 17, 2010)

David MacLeod said:


> would be curious to see what browsers in use if anyone has issue too.
> some people, we won't mention them by name so as to not smoke them out, still use IE8.


As I posted earlier I've experienced issues using Internet Explorer 10 and Google Chrome.


----------



## RAD (Aug 5, 2002)

peds48 said:


> Do not know if it has anything to do with this, but iPhone app does not load


I mentioned earlier that' the iPad was hanging but iPhone OK with the DBSTalk app, this AM the iPhone app is also hanging up.


----------



## scoop8 (Jul 18, 2009)

David MacLeod said:


> would be curious to see what browsers in use if anyone has issue too.
> some people, we won't mention them by name so as to not smoke them out, still use IE8.


 :lol: I'm close to that with IE9, as I finally allowed Windows 7 to install IE10 last night.

I haven't seen any site issues with IE9 but I'm only at the site twice daily, at about the same times.

This isn't related to the thread but I've seen something change from IE9 to 10 this morning. When I was using IE9 at this site, I could use my "back" browser button (I use the "Backspace" key, does the same) and my "first unread" indicator black dots would be automatically refreshed. In other words, I would click on a thread's "first unread" black dot, read the unread posts, then use the browser back-page, and my unread black dots would be refreshed.

That's not happening with IE10. I'm not sure if that's a "cookie" setting in IE10 or something similar related to the IE9-to-10 overnight upgrade that was downloaded.

It's a non-issue for the most part, since I have one of my keys mapped as "ctrl-w" or F5, to refresh with one key.


----------



## David Bott (Jul 12, 2005)

RAD said:


> I mentioned earlier that' the iPad was hanging but iPhone OK with the DBSTalk app, this AM the iPhone app is also hanging up.


Hi...I want to turn off the Sphinx search engine and see if that helps this issue. But I want to be sure you are on-line as I do not want to keep it off for hours.  Let me know if you are around.


----------



## RAD (Aug 5, 2002)

David Bott said:


> Hi...I want to turn off the Sphinx search engine and see if that helps this issue. But I want to be sure you are on-line as I do not want to keep it off for hours.  Let me know if you are around.


How about we schedule it for 13:00 CDT?


----------



## David Bott (Jul 12, 2005)

Well, I just had a report...

"Only have my iPhone with me right now, but the app works fine for me. I'm sending you this reply via the app. Left my iPad at home today. Might be able to borrow one here at work if you need me to."

So not sure if their is an "issue" per say. Hummm...Running the current version?

Oh...this should really be in it's own thread as this thread is really about the slowdowns. Sorry.


----------



## David MacLeod (Jan 29, 2008)

did you get any chance to see how stuff acted during the 4am sphinx update?
curious if you could see what impact there was.


----------



## David Bott (Jul 12, 2005)

I was not up at 4AM...But their was a slow query at that time that was in the log for 117 seconds.


----------



## David MacLeod (Jan 29, 2008)

WTF ?
not up at 4?
LOL

not awful bad....but not great. been wondering if doing it every 12 hrs would actually work better. was wondering if 2 brief slows would be better then the 1 long one. I just don't know if it would be faster or not.
reminds me I need to enable slow logs on mine for sql and php-fpm too, keep forgetting.


----------

