Skip navigation
Currently Being Moderated

Dev Chat 04/28/2011

Created on: Apr 28, 2011 12:04 PM by Nick Yeates - Last Modified:  Apr 28, 2011 12:05 PM by Nick Yeates

[10:46]  * fragfutter activates backup irc logger, to prepare dev session
[10:47]  * Simon4 also has #zenoss logged now
[10:47]  <fragfutter> Simon4: but we still need to tease nyeates with it
[10:48]  <Simon4> fragfutter: we sure do
[10:48]  * Simon4 has plans to do some more 3.0 zenpack coding this weekend, since it's a long one
[10:49]  <fragfutter> it's a long one?
[10:49]  <Simon4> here in the UK yeah
[10:49]  <Simon4> fri-mon off
[10:49]  <Simon4> some silly wedding tomorrow
[10:49]  <fragfutter> ah royal-wedding-holidays
[10:49]  * siXy (~siXy@ has left #zenoss
[10:49]  * kgoedtel (~kgoedtel@ has joined #zenoss
[10:50]  <nyeates> as long as zenoss-logger is here...all good
[10:51]  <fragfutter> nyeates: you won't come off that cheap.
[10:51]  <fragfutter> nyeates: why did no one use zenoss to monitor the zenoss-logger
[10:51]  * mrchippy (~mrchippy@ has joined #zenoss
[10:51]  <nyeates> We really should, its on a restart loop for now
[10:52]  <nyeates> thing is, I think its custom code and the process was still running, but no user logged on
[10:53]  <nyeates> hard to detect zombied applications
[10:54]  * mf2hd ( has joined #zenoss
[10:54]  <rmatte> just use an eggdrop with a logger script
[10:54]  <rmatte> much simpler
[10:55]  <nyeates> not sure what you mean by that rmatte
[10:56]  <rmatte> an eggdrop is an IRC bot that runs as a daemon on a linux box
[10:56]  <rmatte> most popular one out there, been around for ages
[10:56]  <rmatte> it supports TCL scripts
[10:56]  <rmatte> and there are several scripts that provide logging capabilities
[10:56]  <fragfutter> (and it's written in tcl if i remember correctly)
[10:56]  <rmatte> though if your script automagically posts the logs to the website then that's different
[10:56]  <rmatte> I don't think it's actually written in tcl, it's c++
[10:57]  <nyeates> it seperates the logs into days and months, but i have to copy over manually
[10:57]  <rmatte> I'll set one up for the channel later
[10:57]  <nyeates> ill look into eggdrop
[10:57]  <Sam-I-Am> geh, you know your san storage is crap when running scp kills it
[10:57]  <nyeates> o cool
[10:57]  <rmatte> nyeates: I have a lot of experience with eggdrop, I'll set one up on one of my servers
[10:58]  <fragfutter> Sam-I-Am: or your CPU is way oversized (handles scp decryption faster then the san)
[10:58]  <nyeates> Nice, thanks rmatte
[10:58]  <rmatte> hmmm, I wonder if this nick will be taken...
[10:58]  * rmatte is now known as zenny
[10:58]  * zenny is now known as rmatte
[10:58]  <Sam-I-Am> heh
[10:58]  <rmatte> bah, registered
[10:58]  <Sam-I-Am> i remember when i put zenoss on the san and it died horribly
[10:58]  <rmatte> I'll probably name it ZennyBot or something
[10:58]  <Sam-I-Am> now its running on the local storage inside the VM boxes
[10:58]  <fragfutter> zenia?
[10:58]  <Sam-I-Am> you guys fixing the logbot problem?
[10:59]  <Sam-I-Am> fragfutter: xenia! (great tornado, btw)
[10:59]  <nyeates> rmatte: couldnt the eggdrop die just the same? i mean ours didnt die for a year, and all a sudden, bam
[10:59]  * ssureshot (~digitolx@ has joined #zenoss
[11:00]  * themactech (~themactec@ has joined #zenoss
[11:00]  * nyeates has changed the topic to: "Dev session is ON - ask away! - Spring Into Bling ZenPack Event -"
[11:01]  <Sam-I-Am> nyeates: is it on a crappy server?
[11:01]  <Sam-I-Am> fleanode isnt the most stable
[11:01]  <rmatte> nyeates: I've never seen an eggdrop just die in my life
[11:01]  <rmatte> nyeates: not unless the server actually goes down lol
[11:02]  <nyeates> We have Joseph Hanson with us today. Anyone with thoughts on ZenPacks they are going to upgrade?
[11:02]  <rmatte> Apparently Shane Scott has an updated version of the IP-SLA pack already, he just isn't releasing it yet
[11:02]  <rmatte> I'm trying to get him to send it to me
[11:02]  <rmatte> lol
[11:02]  <fragfutter> nyeates: i could point at my gitpull request, still sitting there (and working with zenoss3)
[11:03]  * kocolosk is now known as kocolosk|away
[11:03]  * rocket (~rocket@ has joined #zenoss
[11:03]  * ChanServ gives channel operator status to rocket
[11:03]  <rmatte> yeh, git is becoming a bad system for the community since people aren't responding to the requests
[11:03]  <rmatte> might as well just switch it back to svn and call it a day
[11:03]  <rmatte> lol
[11:03]  <nyeates> Yeah Im sure we can work with Shane rmatte, he is a standup guy
[11:03]  <Sam-I-Am> sup rocket
[11:03]  <rmatte> nyeates: cool
[11:03]  * Simon4 has the BladeChassis ZenPack 99% done for v3 now
[11:04]  <davetoo> rmatte: I'd rather fix the Git process
[11:04]  <nyeates> and i think Shane is in here today :-) yes?
[11:04]  <rocket> keepin the dream alive ..
[11:04]  <rmatte> davetoo: I was joking lol
[11:04]  * Simon4 isn't hating git
[11:04]  <rmatte>
[11:04]  <rocket> rmatte: whats going on with git?
[11:04]  <davetoo> but.. it is definitely self-hangin'-rope if not used carefully
[11:04]  <rmatte> [11:01am] <fragfutter> nyeates: i could point at my gitpull request, still sitting there (and working with zenoss3)
[11:04]  <rmatte> requests aren't being handled
[11:04]  <rocket> oh so a process issue on our side it would seem
[11:05]  * rmatte nods
[11:05]  <Hackman238> Yes, I'm in.
[11:05]  <fragfutter> i thought the release would be done in a few days, after it took me months to get a "go" from my legal department...
[11:06]  <rocket> nyeates: who is handling the commits on our side now?
[11:06]  <rmatte> Hackman238: aha, it's you... couldn't put the name to a nick until now
[11:06]  <nyeates> git: It has been a transition thing after Matt Ray left. Dev team kinda took it, but its been confusing.... I am going to be trying to take it over, but dont have it yet
[11:06]  <Hackman238> rmatte: This weekend I'll get you the source for the IPSLA V3 Zenpack.
[11:07]  <rmatte> Hackman238: thanks, appreciate it
[11:07]  <rmatte> I have a big IPSLA project coming up, and I was going to do a big rewrite of the V2 pack, but no sense in re-inventing the wheel
[11:07]  <Hackman238> rmatte: You can help dev it and test, etc. But I'd rather not release it right now.
[11:07]  <rmatte> Hackman238: Does the v3 pack work with 2.5, or only 3.x?
[11:08]  <rmatte> Hackman238: that's fine, I'm good with just deving/testing for now
[11:08]  <rmatte> until it's more polished
[11:08]  <Hackman238> rmatte: The new version has its own daemon to handle all the work. It only works with V3, but only for GUI reasons. Easily back portable.
[11:08]  <rmatte> Ah, I see
[11:08]  <rmatte> I'd definitely be doing a backport
[11:08]  <rmatte> since I'm still running 2.5 in prod
[11:08]  <rmatte> haven't had time to properly plan and execute an upgrade
[11:09]  <rmatte> Well... either that or I can just continue the route I was going with 2.5, making it work with zenperfsnmp for now
[11:09]  <Hackman238> rmatte: Shouldn't be too hard. My customers run 2.5.2 and 3.0, but the demand for IPSLA lies with the 3.0 customers.
[11:09]  <rmatte> ah
[11:10]  <gwb235> to go from 2.5.2 to 3.1 do we have to do a "mini-jump" to 3.0 or can we go straight to 3.1 ?
[11:10]  <rmatte> gwb235: straight
[11:10]  <gwb235> that's what i figured, but the install docs didn't have that listed IIRC
[11:10]  <rmatte> 3.1 isn't really a new release... it's just a bugfix for 3.0.x
[11:10]  <Hackman238> Also, if anyone is interested, I have a zenpack comming up to handle configuring multiplication of, load balancing, and failover of zopes from inside the webui
[11:11]  <fragfutter> Hackman238: scary
[11:11]  <Hackman238> Makes for very responsive and robust webUI and its very very safe
[11:12]  * Simon4 runs many zopes, it's great
[11:12]  <Hackman238> I've implemented it here at rackspace everywhere, and its working great.
[11:12]  <Simon4> improves response times in the UI heaps with multiple users
[11:12]  * pkw1 (~Adium@ has joined #zenoss
[11:13]  <Hackman238> Absolutely, additionally, it prevents zope lockup due to stupid users who hold F5 or create multi-graph reports with over 9000 items
[11:13]  <Hackman238> Plus failover so you can restart individual zopes with out people having a fit.
[11:14]  <nyeates> gwb235: you can go straught from 2.5.2 to 3.1, but its not tested. Technically, go from 2.5.2 to 3.0.3 to 3.1
[11:14]  <Hackman238> Sticky sessions is a must, however, to minimize duplicate caching and duplicate sessions.
[11:14]  <nyeates> depends how much risk you want to take
[11:14]  * kocolosk|away is now known as kocolosk
[11:15]  <rmatte> Hackman238: that pack only works with enterprise I take it?
[11:15]  <Hackman238> Negative. Will work with core too.
[11:16]  <rmatte> Is there any manual setup required for zope?
[11:16]  <rmatte> or all UI based?
[11:16]  <Hackman238> Negative. All webUI after zenpack install and its post operations.
[11:16]  <rmatte> cool, 3.x specific I take it?
[11:17]  <rmatte> I really need to upgrade my Zenoss instances lol
[11:17]  <Sam-I-Am> me too
[11:17]  <rmatte> but first I need the time to get the LDAP auth working
[11:17]  <Hackman238> Negative, 2.5.x specific, but portable to 3.x.
[11:17]  <Sam-I-Am> rmatte: not working in 3?
[11:17]  <rmatte> then it's going to probably be a 7 hour change to do all my upgrades
[11:17]  <rmatte> Hackman238: ah cool
[11:17]  <rmatte> Sam-I-Am: not yet... the process to get it working is a lot different than it was in 2.5
[11:18]  <Sam-I-Am> oh. how about in enterprise?
[11:18]  <rmatte> Sam-I-Am: enterprise is different, you have that lovely ZenPack that does it for you
[11:18]  <rmatte> lol
[11:18]  <rmatte> speaking of which, I'm fairly certain I could throw together a community ZenPack to do the same
[11:19]  <Hackman238> More zopes =  better so long as 1) users are trapped on a specific zope by cookie using sticky sessions and 2) caching is set reasonable so you do not run out of ram and 3) users dont expire cookies intentionally to get most free threads or specify port in url.
[11:19]  <Hackman238> LDAP auth can be fun....even with the ent. zenpack.
[11:20]  <rmatte> yeh, I got it running perfectly in my 2.5 instances
[11:20]  <nyeates> LDAP was never productized, and it needs to be
[11:20]  <rmatte> just need to sort out installation of the plugin
[11:20]  <themactech> I got LDAP working on 3.1 no prob
[11:20]  <Sam-I-Am> ldap is kinda hackey even in enterprise
[11:20]  <themactech> using ubuntu 10.1 server
[11:21]  <rmatte> themactech: the problem with me is that most of my Zenoss servers are 8.04, which means old system python libraries
[11:21]  <rmatte> so the new LDAP plugin isn't really cooperating.
[11:21]  <themactech> Yeah I gave up on ubuntu running 2.4 python doing LDAP auth
[11:21]  <themactech> So easy once you are running python 2.6
[11:21]  <themactech> very few steps
[11:21]  <rmatte> speaking of which, I threw together an upgrade script to move them from 8.04 to 10.04, I really should run that soon
[11:22]  <Hackman238> I agree with Sam-I-Am. Zenoss issues way too many auth queries, even with auth failure. Results in potiential lockout in the event of 3 strikes and your out LDAP auth policy.
[11:22]  <rmatte> probably 2 days worth of upgrades to do, and they can only be done during our weekly change window, going to take forever
[11:22]  <davetoo> indeed
[11:22]  <rmatte> I think I'll do one next week, so make sure it goes well
[11:22]  <themactech> In my case though I authenticate to Apple OD servers
[11:22]  <rmatte> then continue with one a week until they're all done
[11:22]  <themactech> Have to try with AD soon
[11:23]  <rmatte> themactech: AD works fine from what I hear... I just authenticate against OpenLDAP
[11:24]  <themactech> Yeah AD would give me more options, with OD, since apple uses posix groups, the LDAP plugin doesn't support it so I can't do group assignments.  Some guy has a patch for the python ldap plugin but I don't know how to apply.  It allows the python ldap plugin to handle posix groups
[11:29]  <Sam-I-Am> Hackman238: i havent figured out why its so chatty... and not sure the ram cache helps much.
[11:29]  <Sam-I-Am> Hackman238: my sysadmin has gotten a bit pissed about it
[11:31]  <nyeates> There is a fix posted for too many AD auth request... i poseted it to forum and still working with hackman to get it fixed on his has worked countless other times elsewhere tho. Should be fixed in 3.0
[11:32]  <Hackman238> Hackman238: Same here. My admin doesnt care, but instead, my NOC users end up with a locked SSO which prevents work and creates chaos.
[11:34]  <nyeates> LDAP Auth Fail Fix:
[11:35]  <nyeates> While we have devs here, any questions specific to them?
[11:37]  <Hackman238> I'm sure I'll think of some later. LOL
[11:38]  <Hackman238> Have to run- must interview a guy. Later all; rmatte-I'll PM you the Zenpack this weekend.
[11:41]  <nyeates> Curiosity question. Anyone here using public "cloud" services for any infrastructure? Any Internal "cloud" (self service infra)?
[11:42]  * akafritz (~fritz@ has joined #zenoss
[11:43]  <nyeates> I know Netflix moved to Amazon for much of their infra.... curious to see if others are making any moves
[11:44]  * dubs094 (~dubs@ has joined #zenoss
[11:45]  <nyeates> *crickets* - guess not
[11:45]  <Sashness> my company has been interested in moving towards cloud for certain things but not a full fledge move
[11:45]  <Sashness> at least not yet
[11:45]  <nyeates> yeah, well, that is case with most movers i think
[11:45]  <nyeates> have you looked at any technologies or vendors yet?
[11:46]  <Sashness> biggest issue we have is we made a pretty big investment in hardware/infrastructure which will be rendered moot
[11:46]  <Sashness> i don't think someone wants to justify that part of it. lol
[11:46]  <Sashness> we've looked at using ec2/rackspace/vmware via terremark
[11:46]  <nyeates> your existing resources wont be made moot if you make your own self-service or hybrid cloud that can burst to public
[11:47]  * fragfutter wonders what it will cost to by a petabyte of storage in a cloud
[11:47]  <nyeates> terremark bought by verizon
[11:47]  <Sashness> yeah exactly nyeats. what we want to do is use something like euclyptis so we can be somewhat vendor agnostic and can still use our own virtual infrastructure in house
[11:47]  <Sashness> frag: lol too much
[11:48]  <nyeates> petabye, what is that in TBs?
[11:48]  <Sashness> 1000 TB i believe
[11:48]  <Sashness> 1024 TB
[11:49]  <nyeates> $105,000 to just get 1PB of data into the cloud :-)...nm getting it out and using it
[11:50]  <Sashness> haha
[11:50]  <nyeates> sash: check on compared to eucalyptus - i think its based off openstack/rackspace
[11:51]  <themactech> damn, i feel inadequate, largest SAN we built is 254TB
[11:51]  <nyeates> wow, that is pretty large
[11:51]  <Sashness> nyeates: ohh cool. thanks. will check it out
[11:51]  <akafritz> howdy folks.  does anyone here utilize the community zenpak SQL Data Source?
[11:51]  * davetoo has quit (Quit: Leaving.)
[11:52]  <themactech> We are now testing in our lab building a SAN with the new Promise and Rorke Chassis, with a StorNext MDC instead of Apple's XSAN, since they yanked their xserves
[11:53]  <themactech> All that will have to be monitored by Zenoss
[11:53]  <nyeates> Interesting, how will you do that monitoring themactech?
[11:54]  <themactech> I have made modelers for Apple OS X and for the old Promise VTrak chassis
[11:54]  <themactech> Not where I want them to be yet, still have to figure out how to do custom components in modelers
[11:55]  <themactech> but since SNMP monitoring is part time for me, I get into it, make progress then have to work on something else for a few months...
[11:55]  <themactech> Have to build a holodeck now
[11:55]  <themactech> always busy never bored
[11:59]  <jrh0090> Honestly I have no idea how to that, I can totally ask around though
[12:00]  * zenoss-logger has quit (Remote host closed the connection)
[12:00]  * zenoss-logger (~zenoss-lo@ has joined #zenoss
[12:01]  <nyeates> themactech: i asked jrh if he knew about custom components for modelers
[12:02]  <jrh0090> and I asked kells
[12:02]  <nyeates> I think this will mark the end of our Dev session. Thanks everyone!
[12:03]  * nyeates has changed the topic to: "Spring Into Bling ZenPack Event -"

Comments (0)