Thanks for the information in this topic - got my reports working ...

Ran :

cat /var/clearos/system_database/reports

to generate password

/usr/clearos/sandbox/usr/bin/mysql -u reports -p reports

to enter MySQL

show tables;

brought up tables, but

select * from network; gave me an error

ERROR 1146 (42S02): Table 'reports.network' doesn't exist

exited MySQL,

ran

/usr/sbin/initialize-report-tables network_report yes

to create tables for network report

went back into MySQL, and now I have data


mysql> select * from network;
+----+-------+------------+------------+-----------+---------+---------+------------+------------+-----------+---------+---------+---------------------+
| id | iface | rx_bytes | rx_packets | rx_errors | rx_drop | rx_rate | tx_bytes | tx_packets | tx_errors | tx_drop | tx_rate | timestamp |
+----+-------+------------+------------+-----------+---------+---------+------------+------------+-----------+---------+---------+---------------------+
| 1 | eth0 | 838457332 | 3197794 | 0 | 0 | 9974 | 2946947150 | 3283979 | 0 | 0 | 1018 | 2014-09-08 19:58:32 |
| 2 | eth1 | 45611060 | 406821 | 0 | 0 | 827 | 311551274 | 521216 | 0 | 0 | 9354 | 2014-09-08 19:58:32 |
| 3 | eth2 | 2730017474 | 2911283 | 0 | 0 | 336 | 618243967 | 2383826 | 0 | 0 | 313 | 2014-09-08 19:58:32 |
| 4 | eth3 | 88470 | 788 | 0 | 0 | 0 | 128 | 2 | 0 | 0 | 0 | 2014-09-08 19:58:32 |
+----+-------+------------+------------+-----------+---------+---------+------------+------------+-----------+---------+---------+---------------------+

Now, as to why that happened on two brand new installations, is anybody's guess.

My system log:
error: /usr/clearos/apps/reports_database/libraries/Database_Report.php

Help me plz!


:dry:

Hi Everyone

I have a newly installed system for a week now (6.5.0) and I have the same issue - the Network report shows no data.

I am getting data in all the other reports - load, proxy, network visualizer ...but nothing for the network report.

I tried setting the two values mentioned to 500 and 64 and then ran the three commands to re-initialise, but same result.

Is there anything else I can try, or any other information I can supply to help resolve this.

Many thanks, David

So load values were not coming down. Decided to remove the network detail report and network map apps. Did the 3 commands mentioned earlier in the thread. A days worth of load looks better. Will give this some time and re-install apps.

Are we sure that the sandbox/my.cnf is the problem?

I had a sandbox my.cnf dated Dec 31 2013, the day I setup the server, with 500/64 values. Have not turned off updates, and aside from apps via the dashboard and some experiments with adzapper have not touched things.

Could an unfortunately timed shutdown for electrical work/ups install in May have caused this?

Anyway, changing the values from 500/64 to 128/32 followed by issuing the stop/mv/start commands still sees system-mysqld taking over the cpu at 100% on an intermittent basis.

I will check in a day to see if the 1/5/15 minutes load values are reduced...

Brian.

The warning message is harmless, but we'll look into the the network detail report issue. The "External IP Report" was causing too much MySQL load, and an emergency fix was required.

Sorry to resurrect this, but following the updates to:
May 17 01:47:15 Updated: 1:app-network-detail-report-core-1.5.26-1.v6.noarch
May 17 01:47:15 Updated: 1:app-network-detail-report-1.5.26-1.v6.noarch

It looks like the network detail reports have stopped working.

Looking in the sql database network_detail, its empty.

by running /usr/sbin/networkdetail2db

I get

PHP warning: inet_pton(): Unrecognized address #::1 - /usr/clearos/apps/network/libraries/Network_Utils.php (338)

You could try re-initialising the database or reinstalling the reports with:

yum reinstall app-proxy-report app-proxy-report-core

Thanks i have tried that and there is no data at all under Reports - Filter and Proxy reports see pic.
also when i click on Top IP's - Full Report, it says Nothing to report and under report date it says
No data available in table

I did try this earlier and it did the same and the after about an hour under the Filter and Proxy Report
i did get the graphs and IP addresses , but as i clicked on an IP address I got "No data available in tablet"

is there a way of uninstalling the Wb Proxy repoting program and reinstalling to overwrite the damaged code ?

Terry

On your non-working system, execute the last 3 commands here. If it then works you can delete the file you have moved to /tmp.

I have 2 systems, my own that has been running for about 5 months with full reporting and a new one about 1 month that says "nothing to report" and "no data available in table"

they were both installed from the same media and updated automatically after install

I have gone through the posts in this thread with both systems on display and cant find any differences in the files or logs mentioned

it is possible that another package was installed in a different order in the 2 systems

Is there a way that i could uninstall the filter and proxy reporting and reinstall again

thanks for your help

Terry

I haven't been using ClearOS for long and the problem was not there when I installed but was there after I did the update, so it might be the case that the last batch of updates caused the problem.

Also, I don't know how quick people are at updating their system so an update in May might still be only being discovered now.

Thanks,

Ian.

Yup - I can certainly duplicate the issue by simulating a lag in software updates. This issue might impact users who:

- had the test repositories enabled, or
- disabled software updates and did not do a manual update for a span of time in May/June

I'll push out a high-priority fix in the next 24 hours. Though the issue impacts a relatively smallish number of users, it's certainly a show stopper for those users.

Something is definitely not right. The system database updates were published in late May, so it's interesting that we're only seeing these issues crop up in the last few days. I'll be working on today.

I entered these values, restarted the databases, and after about 30 minutes, the reports started working again. :woohoo:

I, for sure, have never modified these files. Given that the reports used to work, I suspect that these values were set automatically after an update which subsequently broke the report.

Thanks for your help. Really happy to have it all working now.

Regards,

Ian.

The values you have are from the basic system-mysql installation. There must be some package that I have which updated them to:

innodb_log_file_size=500M

innodb_log_buffer_size=64M

You could try either set of values, but whatever you do, afterwards execute the commands:

service system-mysqld stop

mv /var/lib/system-mysql/ib_logfile* /tmp

service system-mysqld start

I have this problem also. I checked the values in the conf file and they show: -

innodb_log_file_size=128M
innodb_log_buffer_size=32M

What should they be set to?

I have a very basic installation running almost no services other than DNS and DHCP on a VM with 1GB or RAM.

Thanks,

Ian.

All done, thanks Nick.

I see what happened. If the system-mysql update is installed before the app-system-database-core update, then the my.cnf file will change (*** see note below). That ordering issue can't happen on new installs because of the install wizard, but it certainly could have happened if:

- One of the test repos was used at some point

- Software updates were disabled over a certain time period and the app-system-database-core update (released weeks before the system-mysql update) was never installed

With the bug reported by other people as well including a vanilla installation

For vanilla installs: the handful of times that I have seen reports fail was due to lack of RAM (512M or 768M of RAM, or something like 1 GB with the kitchen sink running). The system database won't even initialize if there's not enough RAM and this is especially acute with Zarafa's non-trivial RAM requirements. This is one of the reasons we added this tracker item:

Add warning in wizard if inadequate RAM is detected

A user should know if RAM is insufficient ... very bad things happen when that's the case!


*** this is funky but understandable RPM behavior. If a configuration file has changed, an RPM install will not overwrite the file. However, if a configuration file has not changed, an upgraded configuration file will take its place. I understand the reasoning behind this logic, but it causes surprises and grief when it comes to software maintenance.

To my knowledge I've never edited that file. With the bug reported by other people as well including a vanilla installation, I guess it is or was an update that caused it or perhaps the installation of another package which many people don't normally run?

Thanks for troubleshooting. Can I remove access now or do you need it a bit more?

Thanks Nick.

The issue was likely due to a change in the /usr/clearos/sandbox/etc/my.cnf.

  innodb_log_file_size=x

  innodb_log_buffer_size=y

If you ever change these parameters, the ib_logfile0 and ib_logfile1 files /var/lib/system-mysql/ need to be reset or the database bets unhappy:

  service system-mysqld stop

  mv /var/lib/system-mysql/ib_logfile* /tmp

  service system-mysqld start

The default innodb values in ClearOS were changed, but the upgrade should leave any old configuration files alone. I'm not sure if there was an upgrade issue of some kind, or just a manual edit.

Pete,
PM and e-mail sent.
Nick

Pete,
I can help if you want. My gateway shows the symptoms in both Firefox and IE. I'll be going offline in the next half hour or so so it may be better tomorrow evening UK time. I'm happy to do either an IPSec VPN or OpenVPN link.
Nick

Interesting. The data is in the database, so it's likely a PHP or -- even more likely -- a Javascript/Ajax issue. What web browser are you using?

With the Canada Day (July 1) and US Independence Day (July 4) last week, there was less time to handle some of the forum/Community requests. I'm on it now!

Does someone have a box that still has this problem? And can I get remote access :-)

Manual route, "select * from network;" gives:

... big snip ...

| 2776 | eth1  | 12662983331 |   41868500 |         0 |       0 |     451 | 64386582774 |   56278099 |         0 |       0 |    3289 | 2013-06-19 20:30:01 |

| 2777 | eth2  |   356082336 |    4423355 |         0 |       0 |       0 | 11240095111 |    8775866 |         0 |       0 |       0 | 2013-06-19 20:30:01 |

| 2778 | eth0  | 75619607046 |   79780221 |         0 |       0 |    3621 | 16505552656 |   56003926 |         0 |      80 |     412 | 2013-06-19 20:30:01 |

| 2779 | tun0  |    44688857 |     101660 |         0 |       0 |       0 |     7754232 |      84651 |         0 |       0 |       0 | 2013-06-19 20:30:01 |

| 2780 | tun1  |           0 |          0 |         0 |       0 |       0 |           0 |          0 |         0 |       0 |       0 | 2013-06-19 20:30:01 |

| 2781 | eth1  | 12664035145 |   41874773 |         0 |       0 |    3494 | 64395341694 |   56286099 |         0 |       0 |   29099 | 2013-06-19 20:35:02 |

| 2782 | eth2  |   356082336 |    4423355 |         0 |       0 |       0 | 11240095111 |    8775866 |         0 |       0 |       0 | 2013-06-19 20:35:02 |

| 2783 | eth0  | 75628310652 |   79788262 |         0 |       0 |   28916 | 16506458407 |   56009128 |         0 |      80 |    3009 | 2013-06-19 20:35:02 |

| 2784 | tun0  |    44688857 |     101660 |         0 |       0 |       0 |     7754232 |      84651 |         0 |       0 |       0 | 2013-06-19 20:35:02 |

| 2785 | tun1  |           0 |          0 |         0 |       0 |       0 |           0 |          0 |         0 |       0 |       0 | 2013-06-19 20:35:02 |

+------+-------+-------------+------------+-----------+---------+---------+-------------+------------+-----------+---------+---------+---------------------+

2785 rows in set (0.01 sec)

So there is data there and it therefore seems to be a PHP issue. I don't know where to begin with that.

You can inspect the contents of system-mysql graphically by using phpmyadmin - see my post here
http://www.clearfoundation.com/component/option,com_kunena/Itemid,232/catid,39/func,view/id,41438/limit,10/limitstart,60/#42525

Or from the command line...

cat /var/clearos/system_database/reports

#extract password, login to mysql console

/usr/clearos/sandbox/usr/bin/mysql -u reports -p reports

mysql> show tables;

+-------------------+

| Tables_in_reports |

+-------------------+

| network           |

| proxy             |

| proxy_domains     |

| resource          |

+-------------------+

4 rows in set (0.00 sec)



mysql> select * from network;

...SNIP...

| 130585 | eth0  |  7836163422 |   26799753 |         0 |       0 |   43990 |  43043335730 |   23390470 |         0 |       0 |    99756 | 2013-06-18 23:30:02 |

| 130586 | eth1  |  1874211337 |    4858268 |         0 |       0 |    1759 |   6514809647 |    4781784 |         0 |       0 |    42142 | 2013-06-18 23:30:02 |

| 130587 | eth0  |  7836846508 |   26805969 |         0 |       0 |    2277 |  43051716231 |   23398523 |         0 |       0 |    27935 | 2013-06-18 23:35:02 |

| 130588 | eth1  |  1874734261 |    4860342 |         0 |       0 |    1743 |   6515506647 |    4783601 |         0 |       0 |     2323 | 2013-06-18 23:35:02 |

+--------+-------+-------------+------------+-----------+---------+---------+--------------+------------+-----------+---------+----------+---------------------+

130588 rows in set (0.47 sec)

do you have any data in your database? If so, then the problem points at the PHP app implementation...

I still have empty reports

How do I look at the database? Presumably it is in system-mysql?

Thanks, Tim.

[root@server ~]# /usr/sbin/initialize-report-tables network_report yes

creating tables for network_report

[root@server ~]# /usr/sbin/network2db

[root@server ~]#

I'll have to post back tomorrow evening with the results

Hmm you can force it to drop and recreate by adding 'yes'

/usr/sbin/initialize-report-tables network_report yes

If you manually run network2db does it complete without error?

I get the following:

[root@server ~]# /usr/sbin/initialize-report-tables network_report

tables already exist for network_report

The cron script is in cron.d and every 5 minutes I have an entry in the cron log:

Jun 17 18:05:01 server CROND[336]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1)

I am wondering if I need to reboot the server to complete the upgrade to 6.4 but I've been putting it off for the moment for various reasons.

p.s you should have this in /etc/cron.d/app-network-reports

*/5 * * * * root /usr/sbin/network2db >/dev/null 2>&1

and entries in /var/log/cron
Jun 17 14:05:01 leonardo CROND[1857]: (root) CMD (/usr/sbin/network2db >/dev/null 2>&1)

Hmm, a background script (/usr/sbin/network2db) periodically collects data and stores it in system-mysql

The app should install by running the following

/usr/sbin/initialize-report-tables network_report

You can try and re-run this command from the command line to drop and recreate the table?

What happens if you manaully run /usr/sbin/network2db?

@maxwell,
Do you have the repo clearos-core eabled by default?

Thanks for pointing it out. My dashboard is also broken like yours.

I've no idea how to troubleshoot this one as I don't do php. Anyone?

I also do not see

Thanks for the replies. I've had 1.4.30 since 10/04 so I wonder what is going on. I'll try reinstalling it for a start.

Hi Nick,

Mine is working. I have version:

app-network-report.noarch                            1:1.4.30-1.v6                   @anaconda-ClearOSCommunity-201303291439.x86_64/6.4.0

app-network-report-core.noarch                       1:1.4.30-1.v6                   @anaconda-ClearOSCommunity-201303291439.x86_64/6.4.0

Hi Nick,

I had the same issue just after installing the app.
After an hour or so, it started to report.
If this is not the case, then something else might be going on.

Greetings,

John