Michael's Blog http://www.watters.ws News Feed How to upgrade Spacewalk database schema http://www.watters.ws/view/article/191 2018-05-30 12:44:00 <p>After installing updates for spacewalk you may see a message like this on the home screen.</p> <tt>A schema upgrade is required. Please upgrade your schema at your earliest convenience to receive latest bug fixes and avoid potential problems. </tt> <p>To fix this run the following commands.</p> <pre class='code'> spacewalk-service stop spacewalk-schema-upgrade spacewalk-service start </pre> Happy Thanksgiving! http://www.watters.ws/view/article/177 2017-11-21 15:46:00 <p>Another year, another turkey. :D</p> <pre> ,+*^^*+___+++_ ,*^^^^ ) _+* ^**+_ +^ _ _++*+_+++_, ) _+^^*+_ ( ,+*^ ^ \+_ ) { ) ( ,( ,_+--+--, ^) ^ { (@) } f ,( ,+-^ __*_*_ ^^\_ ^\ ) {:;-/ (_+*-+^^^^^+*+*<_ _++_)_ ) ) / ( / ( ( ,___ ^*+_+* ) < < U _/ ) *--< ) ^\-----++__) ) ) ) ( ) _(^)^^)) ) )\^^^^^))^*+/ / / ( / (_))_^)) ) ) ))^^^^^))^^^)__/ +^^ ( ,/ (^))^)) ) ) ))^^^^^^^))^^) _) *+__+* (_))^) ) ) ))^^^^^^))^^^^^)____*^ \ \_)^)_)) ))^^^^^^^^^^))^^^^) (_ ^\__^^^^^^^^^^^^))^^^^^^^) ^\___ ^\__^^^^^^))^^^^^^^^) ^^^^^\uuu/^^\uuu/^^^^\^\^\^\^\^\^\^ ___) >____) >___ ^\_\_\_\_\_\_\) ^^^//\_^^//\_^ ^(\_\_\_\) ^^^ ^^ ^^^ ^^ # # ## ##### ##### # # # # # # # # # # # # ###### # # # # # # # # # ###### ##### ##### # # # # # # # # # # # # # # # ##### # # ## # # # # #### # # # # # ## # # # # # ###### # # # # # #### #### # # # ###### # # # # # # # # # # # # ## # # # # # # # # # # # # # #### #### # # # # # # #### # # # # # # ## # # # # # # # # # # # # # ### # # # # # # # # ### # # # # # # # ## # # #### # ## # # # #### </pre> <p>Source code available on <a href='https://gist.github.com/blackknight36/b9f697be48f2301de992b037cf66e74a'>github</a>.</p> How to fix graphite user creation in FreeBSD 10 http://www.watters.ws/view/article/190 2016-03-09 14:22:00 <p>If you see an error like below when you try to create a graphite user you will need to update the graphite database in order for user create to work. <p class='code'> django.db.utils.IntegrityError: NOT NULL constraint failed: auth_user.last_login </pre> <p>Unfortunately sqlite doesn't support the drop constraint syntax so you will need to create a new temporary table, copy over the existing table, and then rename. <pre class='code'> CREATE TABLE "auth_user2" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "password" varchar(128) NOT NULL, "last_login" datetime NULL, "is_superuser" bool NOT NULL, "username" varchar(30) NOT NULL UNIQUE, "first_name" varchar(30) NOT NULL, "last_name" varchar(30) NOT NULL, "email" varchar(75) NOT NULL, "is_staff" bool NOT NULL, "is_active" bool NOT NULL, "date_joined" datetime NOT NULL); insert into auth_user2 select * from auth_user ; drop table auth_user ; alter table auth_user2 rename to auth_user ; </pre> <p>Now user creation should work. <pre class='code'> root@graphite:/usr/local/lib/python2.7/site-packages/graphite # python manage.py createsuperuser Username (leave blank to use 'root'): wattersm Email address: wattersm@watters.ws Password: Password (again): Superuser created successfully. </pre> Selecting random records in postgresql http://www.watters.ws/view/article/172 2015-08-18 09:15:00 <p>In an attempt to make my site more responsive I have been working on optimizing the SQL code used on the backend, this includes the random quote generator that I have set up on the main page.</p> <p>The old code used a query similar to below.</p> <p class="code">SELECT quote, name FROM quotes ORDER BY RANDOM() LIMIT 1</p> <p>This works fine if you have small tables and fast disks but consider the issue when there is a table with millions of rows. To find *one* record the server must read through the table, sort the records, and then discard every result but one. This operation is slow and inefficient.</p> <p>To improve performance you can reduce the number of rows read by using a primary key on the table. Each row has a unique ID number which can then be used as the limit for the random() function. For example, the following query will select a random record based on the last index number created:</p> <p class='code'>SELECT quote_text, name FROM quotes WHERE quote_id = (SELECT floor(random() * (SELECT last_value from quotes_quote_id_seq)+1)) ; </p> <p>This query is not perfect and may result in empty results which your code will need to accommodate for but it is still more efficient than reading the entire table every time the page is loaded.</p> Prevent iptables from spamming your console http://www.watters.ws/view/article/193 2015-08-14 13:41:00 <p>How to disable firewall "spam" on your console.</p> <p>I worked on a ticket recently for a customer concerned about firewall messages being sent to every user's console by the kernel. After doing a bit of research I discovered that the nf_ct_ftp module logs messages to syslog as *emergency* level by default which results in every console being spammed by firewall messages. To prevent this you can make a few simple changes as follows.</p> <p>First, set up a custom rsyslog conf file to send iptables messages to a different file.</p> <pre class='code'> cat << EOF > /etc/rsyslog.d/iptables.conf :msg, contains, "nf_ct_ftp:" -/var/log/iptables.log & ~ EOF </pre> <p>The first line means send all messages that contain the “nf_ct_ftp:” string to /var/log/iptables.log. The second line causes rsyslog to discard messages that were matched on the previous line. Adjust this rule according to your needs.</p> <p>Second, update sysctl.conf with the following lines and then run "sysctl -p".</p> <pre class='code'> kernel.printk = 4 4 1 7 sysctl -p </pre> <p>See https://www.kernel.org/doc/Documentation/sysctl/kernel.txt for a description of these values.</p> <p>Now restart rsyslog and test your changes using the "logger" command.</p> <pre class='code'> service rsyslog restart logger -p kern.emerg -t kernel "nf_ct_ftp: dropping packet test" </pre> <p>You should not see anything on the console. cat /var/log/iptables.log to confirm that the entry was logged properly. After you have confirmed that the messages are being logged properly you can set up logrotate to manage the logs. Create a config file to do this similar to below.</p> <pre class='code'> cat << EOF > /etc/logrotate.d/iptables /var/log/iptables.log { rotate 7 daily missingok notifempty delaycompress compress postrotate invoke-rc.d rsyslog rotate > /dev/null endscript } EOF </pre> <p>There is nothing else to do at this point.</p> How to Manually Change Domain in Magento http://www.watters.ws/view/article/189 2014-10-10 10:20:00 <p>Changing the domain name on a Magento install requires a few steps to update the site URL in mysql. The procedure should be similar to below.</p> <p>Update your core_config_data table to edit the two records for <i>web/unsecure/base_url</i> and <i>web/secure/base_url</i></p> <pre class='code'> mysql <database> mysql> update core_config_data set value = 'http://dev.example.com/' where path = 'web/unsecure/base_url'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> update core_config_data set value = 'http://dev.example.com/' where path = 'web/secure/base_url'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 </pre> <p>After this is done delete the contents of WEBROOT/var/cache. The location of the WEBROOT varies depending on how your server is set up.</p> <pre class='code'> cd /home/username/public_html/var/ rm -rf ./cache/* </pre> <p>Update any .htaccess redirects you may have added.</p> <p>That's it, you're done! Open the site in a new browser tab to make sure that everything loads properly.</p> How to fix "Your profile could not be loaded" error in Google Chrome http://www.watters.ws/view/article/188 2014-03-20 21:04:00 <p>If you get an error from chrome stating that your profile could not be loaded properly here is the PROPER way to fix the issue. Unfortunately googling for this error leads to a lot of false information and speculation.</p> <p>First, go to your profile's data directory. In Linux this would be ~/.config/google-chrome/Default.</p> <p>Now check for any processes that have the Web Data file open.</p> <p>lsof Web\ Data</p> <p>Kill those processes.</p> <p>Next run an integrity check on the database.</p> <p class='code'>sqlite3 Web\ Data "pragma integrity_check"</p> <p>This should repair any errors in the file. After that is done start up chrome.</p> Canola Soap Recipe http://www.watters.ws/view/article/187 2013-09-15 09:47:00 <p>Here is a simple canola/soybean oil soap recipe. Yield is approximately 8 lbs of soap. Lye amount is based on a saponification value of 0.130. See http://www.millersoap.com/soapdesign.html#SAP Tables for more details.</p> <p>Canola/Soybean Soap</p> Ingredients: <ul> <li>70 oz canola oil</li> <li>16 oz soybean oil</li> <li>28 oz water</li> <li>11 oz lye</li> <li>Scent - optional</li> <li>Coloring - Use dye or a small piece of crayon.</li> </ul> Preparation: <ol> <li>Dissolve lye into water. Prepare this mixture in advance, the water will take time to cool.</li> <li>Pour oil into large pot, heat to 130 degrees.</li> <li>Stir lye mixture into the oil.</li> <li>Blend with a stick blender until you see signs of tracing.</li> <li>Pour into molds and let harden.</li> </ol> <p>Raw soap will take 24-48 hours to harden, after that the soap can be removed from the mold to cure. Allow 30 days cure time.</p> Manually create a Wordpress admin user from the mysql command line http://www.watters.ws/view/article/186 2013-05-01 16:06:00 <p>If you need admin access to a wordpress install you can easily create a new admin user by running a few SQL commands on the database. This has been tested and verified to work on Wordpress 3.5.</p> <p>To do this you will first need to identify what database the site is actually using. Check wp-config.php for the database name and mysql host info. Once you have that connect to mysql and run the following statements.</p> <p class="code">INSERT INTO wp_users (user_login,user_pass,user_email,user_registered,user_status) VALUES("user_name",md5('password'),"username@example.com",NOW(),0);</p> <p>Find user ID from wp_users table:</p> <pre class="code">SET @user_id = (SELECT ID FROM wp_users where user_login = 'user_name'); INSERT INTO wp_usermeta (user_id,meta_key,meta_value) VALUES (@user_id,"wp_user_level","10"); INSERT INTO wp_usermeta (user_id,meta_key,meta_value) VALUES (@user_id,"wp_capabilities",'a:1:{s:13:"administrator";s:1:"1";}'); </pre> Storm VPS Lustre Benchmarks http://www.watters.ws/view/article/185 2013-04-11 13:56:00 <p>After reading about various cluster file systems I decided to set up a small cluster running Lustre using Storm VPS instances. All nodes have the same hardware configuration and use a 50 GB SAN volume connected through iSCSI as the lustre block device. Specs are as follows.</p> <pre> Node configuration: OS: CentOS 6.3 x86_64 Kernel: 2.6.32-279.19.1.el6_lustre.x86_64 RAM: 3556 MB (Storm 4 GB) Primary Disk: 300 GB virtual disk Secondary Disk (iscsi): 50 GB SAN volume CPU: Two Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz cores Lustre configuration: 1 management server, 1 metadata server, 1 object storage server. LNET was configured to use a private network interface. </pre> <p>Disk performance was tested with the sgpdd_survey script from the Lustre IOkit. Write speed appears to average around 35-40 MB/s.</p> <pre> Wed Apr 10 10:29:39 EDT 2013 sgpdd-survey on /dev/sda from oss1.watters.ws total_size 8388608K rsz 1024 crg 1 thr 1 write 49.32 MB/s 1 x 49.32 = 49.32 MB/s read 68.15 MB/s 1 x 68.15 = 68.15 MB/s total_size 8388608K rsz 1024 crg 1 thr 2 write 77.15 MB/s 1 x 77.15 = 77.15 MB/s read 92.85 MB/s 1 x 92.85 = 92.85 MB/s total_size 8388608K rsz 1024 crg 1 thr 8 write 36.15 MB/s 1 x 36.14 = 36.14 MB/s read 94.08 MB/s 1 x 94.09 = 94.09 MB/s total_size 8388608K rsz 1024 crg 1 thr 16 write 35.84 MB/s 1 x 35.85 = 35.85 MB/s read 101.59 MB/s 1 x 101.59 = 101.59 MB/s total_size 8388608K rsz 1024 crg 2 thr 2 write 35.34 MB/s 2 x 17.67 = 35.34 MB/s read 67.38 MB/s 2 x 33.69 = 67.39 MB/s total_size 8388608K rsz 1024 crg 2 thr 4 write 39.09 MB/s 2 x 19.55 = 39.10 MB/s read 79.20 MB/s 2 x 39.60 = 79.19 MB/s total_size 8388608K rsz 1024 crg 2 thr 8 write 40.40 MB/s 2 x 20.20 = 40.40 MB/s read 98.16 MB/s 2 x 49.09 = 98.17 MB/s total_size 8388608K rsz 1024 crg 2 thr 16 write 37.73 MB/s 2 x 18.86 = 37.73 MB/s read 99.31 MB/s 2 x 49.66 = 99.32 MB/s total_size 8388608K rsz 1024 crg 2 thr 32 write 38.08 MB/s 2 x 19.04 = 38.07 MB/s read 97.30 MB/s 2 x 48.66 = 97.31 MB/s total_size 8388608K rsz 1024 crg 4 thr 4 write 38.38 MB/s 4 x 9.59 = 38.38 MB/s read 98.17 MB/s 4 x 24.55 = 98.19 MB/s total_size 8388608K rsz 1024 crg 4 thr 8 write 38.25 MB/s 4 x 9.57 = 38.26 MB/s read 100.06 MB/s 4 x 25.01 = 100.06 MB/s total_size 8388608K rsz 1024 crg 4 thr 16 write 39.42 MB/s 4 x 9.85 = 39.41 MB/s read 99.96 MB/s 4 x 25.00 = 99.98 MB/s total_size 8388608K rsz 1024 crg 4 thr 32 write 39.43 MB/s 4 x 9.86 = 39.44 MB/s read 99.93 MB/s 4 x 24.99 = 99.95 MB/s total_size 8388608K rsz 1024 crg 4 thr 64 write 38.22 MB/s 4 x 9.56 = 38.22 MB/s read 97.80 MB/s 4 x 24.45 = 97.81 MB/s total_size 8388608K rsz 1024 crg 8 thr 8 write 38.73 MB/s 8 x 4.84 = 38.76 MB/s read 87.71 MB/s 8 x 10.97 = 87.74 MB/s total_size 8388608K rsz 1024 crg 8 thr 16 write 39.70 MB/s 8 x 4.96 = 39.67 MB/s read 81.09 MB/s 8 x 10.14 = 81.10 MB/s total_size 8388608K rsz 1024 crg 8 thr 32 write 43.40 MB/s 8 x 5.43 = 43.41 MB/s read 81.21 MB/s 8 x 10.16 = 81.25 MB/s total_size 8388608K rsz 1024 crg 8 thr 64 write 38.88 MB/s 8 x 4.86 = 38.91 MB/s read 67.10 MB/s 8 x 8.39 = 67.14 MB/s total_size 8388608K rsz 1024 crg 8 thr 128 write 42.19 MB/s 8 x 5.27 = 42.19 MB/s read 65.92 MB/s 8 x 8.24 = 65.92 MB/s </pre> <p>IOPS performance was tested using <i>iozone</i>, here are the results.</p> <pre> OPS Mode. Output is in operations per second. Include fsync in write timing No retest option selected Record Size 4 KB File size set to 4194304 KB Command line used: iozone -l 32 -O -i 0 -i 1 -i 2 -e -+n -r 4K -s 4G Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Min process = 32 Max process = 32 Throughput test with 32 processes Each process writes a 4194304 Kbyte file in 4 Kbyte records Children see throughput for 32 initial writers = 27764.87 ops/sec Parent sees throughput for 32 initial writers = 26692.16 ops/sec Min throughput per process = 840.07 ops/sec Max throughput per process = 903.35 ops/sec Avg throughput per process = 867.65 ops/sec Min xfer = 975918.00 ops Children see throughput for 32 readers = 26758.37 ops/sec Parent sees throughput for 32 readers = 26755.12 ops/sec Min throughput per process = 448.79 ops/sec Max throughput per process = 1372.74 ops/sec Avg throughput per process = 836.20 ops/sec Min xfer = 342845.00 ops </pre> <p>As you can see lustre is a relatively high performance file system and is easily scalable to store petabytes of data. Adding more space is as simple as building a new object server and running mkfs.lustre.</p>