Mysql database query very slow with huge data / rows using CREATE INDEX in PHP

CREATE INDEX visitorhistory_store_created_at_index ON visitorhistory (store, created_at);

The reference use: https://blog.nodeswat.com/making-slow-queries-fast-with-composite-indexes-in-mysql-eb452a8d6e46

How to change root password of mysql

sudo service mysql stop
sudo mkdir -p /var/run/mysqld

sudo chown mysql:mysql /var/run/mysqldsudo /usr/sbin/mysqld --skip-grant-tables --skip-networking &

mysql -u rootmysql> FLUSH PRIVILEGES;

mysql> USE mysql;mysql> UPDATE user SET authentication_string=PASSWORD("<Your Password>") WHERE User='root';mysql> UPDATE user SET plugin="mysql_native_password" WHERE User='root';

mysql> exit
sudo pkill mysqldsudo service mysql start

Re-index primary key in Mysql database table

set @ROW = 0;

UPDATE `visitorhistory` SET `id` = @ROW := @ROW+1 ORDER BY `id` ASC;

How to export database from terminal / CMD in mysql

mysqldump -u root -p databasename > /var/www/html/lospol_3172019.sql

sql_mode=only_full_group_by is incompatible in PHP mysql query with GROUP BY codeigniter

#1055 - Expression #24 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'customers.first_name' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

Add $this->db->query("SET sql_mode=(SELECT REPLACE(@@sql_mode, 'ONLY_FULL_GROUP_BY', ''));") line above your query.

See Example :

$this->db->query("SET sql_mode=(SELECT REPLACE(@@sql_mode, 'ONLY_FULL_GROUP_BY', ''));");
$this->db->select('*, customers.first_name, customers.last_name, customers.id');
$this->db->select("(SELECT count(video_id) FROM customer_videos WHERE c.customer_id = customer_videos.customer_id AND (created_at >= '$from' OR created_at <= '$todate')) as seen");
$this->db->join('customers', 'customers.id = c.customer_id', 'left');
$this->db->group_by('c.customer_id');
$this->db->from('customer_videos as c');
$query = $this->db->get();
echo "<pre/>"; print_r($query->result());exit;
return $query->result_array();

How to log all | full SQL | Mysql queries in Magento 2

Show ALL | Full magento 2 query with Sorting, Pagination and Filter in Magento 2

Open : /var/www/html/magento/jaydip kansagra/app/etc/di.xml

Find 

<preference for="Magento\Framework\DB\LoggerInterface" 

Replace

<preference for="Magento\Framework\DB\LoggerInterface" type="Magento\Framework\DB\Logger\File"/>
 <type name="Magento\Framework\DB\Logger\File">
      <arguments>
          <argument name="logAllQueries" xsi:type="boolean">true</argument>
          <argument name="debugFile" xsi:type="string">log/sql.log</argument>
      </arguments>
 </type>

Open : /var/www/html/magento/frye-magento/var/log/sql.log

Join Query with LIKE in PHP Mysql

Join Query with %LIKE% function in MYSQL PHP

SELECT `customers`.*, `admin`.`firstname` AS `salespersonfname`, `admin`.`lastname` AS `salespersonlname`
FROM `customers`
LEFT JOIN `admin` ON customers.tags LIKE CONCAT('%', `admin`.firstname, ' ', `admin`.lastname, '%')
ORDER BY `id` DESC

SQLSTATE[42S22]: Column not found: 1054 Unknown column '' in 'having clause', query was: SELECT COUNT(*) in Magento 1

Step 1. Open Collection.php of that model : root/app/code/local/Jaydip/Kansagra/Model/Resource/Rebatecash/Collection.php

Step 2.  Create getSelectCountSql() function in Collection.php file
public function getSelectCountSql() {
        $this->_renderFilters();
        $select = clone $this->getSelect();
        $select->reset(Zend_Db_Select::ORDER);
        $select->reset(Zend_Db_Select::LIMIT_COUNT);
        $select->reset(Zend_Db_Select::LIMIT_OFFSET);       
        $countSelect = clone $this->getSelect();
        $countSelect->reset();
        $countSelect->from(array('a' => $select), 'COUNT(*)');
        return $countSelect;
    }

Optimized my.cnf MySQL configuration due to hign CPU usage in Ubuntu

# Optimized my.cnf configuration for MySQL/MariaSQL
#
# by Fotis Evangelou, developer of Engintron (engintron.com)
#
# === Updated July 2018 ===
#
# The settings provided below are a starting point for a 2GB - 4GB RAM server with 2-4 CPU cores.
# If you have less or more resources available you should adjust accordingly to save CPU,
# RAM and disk I/O usage.
# The settings marked with a specific comment or the word "UPD" after the value
# should be adjusted for your system by using database diagnostics tools like:
# https://github.com/major/MySQLTuner-perl
# or
# https://launchpad.net/mysql-tuning-primer (supports MySQL up to v5.6)
#
#
# Note that if there is NO comment after a setting value, then 99,9% of the times you won't need to adjust it.
#
#
# THINGS TO DO AFTER YOU UPDATE MY.CNF - TROUBLESHOOTING
# If any terminal commands are mentioned, make sure you execute them as "root" user.
# If MySQL cannot start or restart, then perform the following actions.
#
# 1. If the server had the stock database configuration and you added or updated any
#    "innodb_log_*" settings (as suggested below), then execute these commands ONLY
#    the first time you apply this configuration:
#
#    $ rm -rvf /var/lib/mysql/ib_logfile*
#    $ chown -R mysql:mysql /var/lib/mysql
#    $ service mysql restart
#
#    or use the shorthand command:
#    $ rm -rvf /var/lib/mysql/ib_logfile*; chown -R mysql:mysql /var/lib/mysql; service mysql restart
#
# 2. If the setting "bind-address" is not commented out, then make sure the file /etc/hosts is
#    properly configured. A good example of a "clean" /etc/hosts file is something like this:
#
#    127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#    ::1       localhost localhost.localdomain localhost6 localhost6.localdomain6
#    1.2.3.4   hostname.domain.tld hostname # Replace accordingly!
#
#    Finally restart the database service:
#
#    $ service mysql restart
#
# 3. If the database service cannot restart even after the first 2 steps, make sure the database data folder
#    (common for either MySQL or MariaDB) "/var/lib/mysql" is owned by the "mysql" user AND group.
#    Additionally, the folder itself can have 0751 or 0755 file permissions. To fix it, simply do this:
#    $ chown -R mysql:mysql /var/lib/mysql
#    $ chmod 0755 /var/lib/mysql
#
#    Finally restart the database service:
#
#    $ service mysql restart
[mysql]
port                            = 3306
socket                          = /var/run/mysqld/mysqld.sock
[mysqld]
# Required Settings
basedir                         = /usr
bind_address                    = 127.0.0.1 # Change to 0.0.0.0 to allow remote servers to connect to this server's
                                            # database instance
datadir                         = /var/lib/mysql
max_allowed_packet              = 256M
max_connect_errors              = 1000000
pid_file                        = /var/run/mysqld/mysqld.pid
port                            = 3306
skip_external_locking
skip_name_resolve
socket                          = /var/run/mysqld/mysqld.sock
#sql_mode                       = ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES # Enable for b/c with databases
                                                                                                                                               # using null dates
tmpdir                          = /tmp
user                            = mysql
# InnoDB Settings
default_storage_engine          = InnoDB
innodb_buffer_pool_instances    = 1     # Use 1 instance per 1GB of InnoDB pool size
innodb_buffer_pool_size         = 1G    # Use up to 70-80% of RAM & optionally check if /proc/sys/vm/swappiness is set to 0
innodb_file_per_table           = 1
innodb_flush_log_at_trx_commit  = 0
innodb_flush_method             = O_DIRECT
innodb_log_buffer_size          = 16M
innodb_log_file_size            = 128M
#innodb_thread_concurrency      = 4     # Optional: Set to the number of CPUs on your system (minus 1 or 2) to better
                                        # contain CPU usage. E.g. if your system has 8 CPUs, try 6 or 7 and check
                                        # the overall load produced by MySQL/MariaDB.
# MyISAM Settings
query_cache_limit               = 4M    # UPD
query_cache_size                = 48M   # UPD
query_cache_type                = 1
key_buffer_size                 = 48M   # UPD
low_priority_updates            = 1
concurrent_insert               = 2
# Connection Settings
max_connections                 = 100   # UPD
back_log                        = 512
thread_cache_size               = 100
thread_stack                    = 192K
interactive_timeout             = 180
wait_timeout                    = 180
# Buffer Settings
join_buffer_size                = 3M    # UPD
read_buffer_size                = 2M    # UPD
read_rnd_buffer_size            = 4M    # UPD
sort_buffer_size                = 4M    # UPD
# Table Settings
# In systemd managed systems like Ubuntu 16.04 or CentOS 7, you need to perform an extra action for table_open_cache & open_files_limit
# to be overriden (also see comment next to open_files_limit).
# E.g. for MySQL 5.7, please check: https://dev.mysql.com/doc/refman/5.7/en/using-systemd.html
# and for MariaDB check: https://mariadb.com/kb/en/library/systemd/
table_definition_cache          = 8000  # UPD
table_open_cache                = 8000  # UPD
open_files_limit                = 24000 # UPD - This can be 2x to 3x the table_open_cache value or match the system's
                                        # open files limit usually set in /etc/sysctl.conf or /etc/security/limits.conf
                                        # In systemd managed systems this limit must also be set in:
                                        # /etc/systemd/system/mysqld.service.d/override.conf (for MySQL 5.7+) and
                                        # /etc/systemd/system/mariadb.service.d/override.conf (for MariaDB)
max_heap_table_size             = 128M
tmp_table_size                  = 128M
# Search Settings
ft_min_word_len                 = 3     # Minimum length of words to be indexed for search results
# Logging
log_error                       = /var/lib/mysql/mysql_error.log
log_queries_not_using_indexes   = 1
long_query_time                 = 5
slow_query_log                  = 0     # Disabled for production
slow_query_log_file             = /var/lib/mysql/mysql_slow.log
[mysqldump]
# Variable reference
# For MySQL 5.7: https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html
# For MariaDB:   https://mariadb.com/kb/en/library/mysqldump/
quick
quote_names
max_allowed_packet              = 64M

Refer From : https://gist.github.com/fevangelou/fb72f36bbe333e059b66 

Mysql usage grater than 100% of CPU in ubuntu 18.04 | 16.04 in AWS

 You need to update server configuration.

1. Open terminal

2. Put command : sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

3. Find following variables
key_buffer_size         = 16M
max_allowed_packet      = 16M
thread_stack            = 192K
thread_cache_size       = 8
query_cache_limit      = 1M
query_cache_size        = 16M
Change To
key_buffer_size         = 160M
max_allowed_packet      = 160M
thread_stack            = 1920K
thread_cache_size       = 80
query_cache_limit       = 1024M
query_cache_size        = 2048M

4. Restart mysql : sudo /etc/init.d/mysql restart

5. Restart apache2 sudo systemctl restart apache2

6. Check CPU usage : ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head

 For more Information please Refer : https://dev.mysql.com/doc/refman/5.7/en/server-configuration.html

SELECT list is not in GROUP BY clause and contains nonaggregated column … incompatible with sql_mode=only_full_group_by

Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'returntr_prod.tbl_customer_pod_uploads.id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

will be simply solved by changing the sql mode in MySQL by this command,

SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''));

OR

set @@global.show_compatibility_56=ON;

OR from CMD

mysql -u root -p mg_cstore < db.sql

This is too works me.. I used this becz in my project there are many Queries like this so just change this sql mode only_full_group_by


For permenant solution in ubuntu 16 and 18

Step1) find and modify the config file my.cnf. Usually it’s in /etc/my.cnf or /etc/mysql/my.cnf.

Step 2) Add following code end of the file
[mysqld]sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION

How to get Missing / Deleted primary key from Mysql / PHP Database

select a.id + 1 RemoveIds
from jaydip_kansagra a
left join jaydip_kansagra b
  on a.id = b.id - 1
where b.id is null
  and a.id < 14481

14481 = Last Primary Key 

Re-Arrange primary key | INDEXING in MYSQL

Re-Arrange primary key | INDEXING in MYSQL

SET @count = 0;
UPDATE `users` SET `users`.`id` = @count:= @count + 1;

Last number of records

ALTER TABLE `jaydipkansagra` auto_increment = 720418;  

How to reset MySQL / phpmyadmin root password on Ubuntu 18.04 Linux

1. The simplest approach to reset MySQL database root password is to execute mysql_secure_installation program and when prompted entering your new root MySQL password:

sudo mysql_secure_installation

New password:
Re-enter new password:

2. Let's stop the currently running MySQL database:

sudo service mysql stop

3. create a /var/run/mysqld directory to be used by MySQL process to store and access socket file:

sudo mkdir -p /var/run/mysqld
sudo chown mysql:mysql /var/run/mysqld 

4. Start manually MySQL with the following linux commands :

sudo /usr/sbin/mysqld --skip-grant-tables --skip-networking &

5. Confirm that the process is running as expected: 

jobs

6. Access MySQL database without password 

mysql -u root

7. First flush privileges MySQL session

mysql> FLUSH PRIVILEGES;

8. The following commands will reset MySQL root password to "root" 

mysql> USE mysql;
mysql> UPDATE user SET authentication_string=PASSWORD("root") WHERE User='root';
mysql> UPDATE user SET plugin="mysql_native_password" WHERE User='root';

9. Exit MySQL session: 

mysql> exit

10. Terminate current mysqld process 

sudo pkill mysqld

11. Start MYSQL database:

sudo service mysql start

12. If all went well you should now be able to login to your MySQL database with a root password: 

mysql -u root --password=root
mysql> exit

shell_exec("mysql -u $dbuser --password='$dbpassword' --host='$sqlhost' $dbname < $file") not working in PHP

I try to execute query from PHP / CMD / Terminal

mysql -u admin --password='jaydip' --host='localhost' 'mg_kansagra' < '/var/www/html/magento/dump/db.sql'

Its show me following error

mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 3167 (HY000) at line 17: The 'INFORMATION_SCHEMA.SESSION_VARIABLES' feature is disabled; see the documentation for 'show_compatibility_56'

ANSWER

Run the following query in your MySQL database:
set @@global.show_compatibility_56=ON;

How to get the final sql / mysql query from the grid includes filters and pagination in Magento

Open Following file

Open MagentoRoot/lib/Varien/Db/Adapter/Pdo/Mysql.php

Update false to true

protected $_debug = true
protected $_logAllQueries  = true

Export Big sql/mysql database using php file

Export Big sql/mysql database using php file

error_reporting(E_ALL ^ E_NOTICE);
ini_set('display_errors', 1);
ini_set('memory_limit', '512M');
$dbinfo = array(
    "host" => 'localhost',
    "user" => 'root',
    "pass" => 'root',
    "dbname" => 'dbname'
);

// Database Config
$sqlhost = $dbinfo["host"];
$dbuser = $dbinfo["user"];
$dbpassword = $dbinfo["pass"];
$dbname = $dbinfo["dbname"];
// filename
$file = date('Ymdhis');
echo shell_exec("mysqldump --add-drop-table -u $dbuser -p$dbpassword`cat /etc/psa/.psa.shadow` $dbname > $file.sql");
//shell_exec("mysql -u $dbuser --password='$dbpassword' --host='$sqlhost' $dbname < $file");
echo 'Finished!<br/>';

Get Address of customer from database Query in Magento PHP Mysql

SELECT
`entity_id` as `entity_id`,
`firstname`.`value` as `First_Name`,
`surname`.`value` as `Surname`,
`telephone`.`value` as `Telephone`,
`country`.`value` as `country`,
`region`.`value` as `region`,
`statecounty`.`value` as `statecounty`,
`city`.`value` as `city`,
`company`.`value` as `company`,
`street`.`value` as `street1`,
`customer_entity`.`created_at`,
`customer_entity`.`updated_at`
FROM
`customer_address_entity_varchar` as `country`
INNER JOIN
`customer_address_entity_varchar` as  `firstname` USING (`entity_id`)
INNER JOIN
`customer_address_entity_varchar` as  `surname` USING (`entity_id`)
INNER JOIN
`customer_address_entity_varchar` as  `telephone` USING (`entity_id`)
INNER JOIN
`customer_address_entity_varchar` as  `region` USING (`entity_id`)
INNER JOIN
`customer_address_entity_varchar` as  `statecounty` USING (`entity_id`)
INNER JOIN
`customer_address_entity_varchar` as  `city` USING (`entity_id`)
INNER JOIN
`customer_address_entity_varchar` as  `company` USING (`entity_id`)
INNER JOIN
`customer_address_entity_text` as  `street` USING (`entity_id`)
INNER JOIN
`customer_entity` USING (`entity_id`)
WHERE
`firstname`.`attribute_id` = 20  &&
`surname`.`attribute_id` = 22  &&
`country`.`attribute_id` = 27 &&
`region`.`attribute_id` = 28 &&
`statecounty`.`attribute_id` = 144 &&
`city`.`attribute_id` = 26 &&
`company`.`attribute_id` = 24 &&
`street`.`attribute_id` = 25 &&
`telephone`.`attribute_id` = 31

How to get the latest record in each group using GROUP BY

In PHP MySQL

SELECT `main_table`.`rebate_amount`, `main_table`.`companyname`, (SELECT target FROM batecash WHERE id = MAX(main_table.id)) AS `maxtarget`, SUM(rebate_amount) AS `rebate_amount`, SUM(invoiceamount) AS `purchase_amount`, MAX(id) AS `max_id`, `main_table`.`rebatetarget`, YEAR(created_at) yr, QUARTER(created_at) qt FROM `batecash` AS `main_table` WHERE (customerid = '5') GROUP BY YEAR(created_at), QUARTER(created_at) ORDER BY `created_at` DESC

 In Magento

$rebatecount = Mage::getModel('rebatereward/batecash')->getCollection()
                    ->addFieldToSelect('rebate_amount')
                    ->addFieldToSelect('companyname')
                    ->addFieldToFilter('orderstatus', array('eq' => 'completed'))
                    ->addFieldToFilter('addressid', array('eq' => $address->getId()))
                    ->addFieldToFilter('customerid', array('eq' => $this->getCustomer()->getId()));

            $rebatecount->getSelect()->columns("(SELECT rebatetarget FROM batecash WHERE id = MAX(main_table.id)) as maxtarget");
            $rebatecount->getSelect()
                    ->columns('SUM(rebate_amount) as rebate_amount')
                    ->columns('SUM(invoiceamount) as purchase_amount')
                    ->columns('MAX(id) as max_id')
                    ->columns('rebatetarget')
                    ->columns('YEAR(created_at) yr, QUARTER(created_at) qt')
                    ->group('YEAR(created_at), QUARTER(created_at)')
                    ->order(array('created_at DESC'));

How to install PHP, apache and mysql in ununtu 14.04

https://www.howtoforge.com/ubuntu-lamp-server-with-apache2-php5-mysql-on-14.04-lts

https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-phpmyadmin-on-ubuntu-12-04

Enable .htaccess

sudo a2enmod rewrite
sudo service apache2 restart
sudo nano /etc/apache2/sites-available/000-default.conf
<Directory "/var/www/html">
AllowOverride All
</Directory>

sudo service apache2 restart