Innodb数据库对于已经删除的数据只是标记为删除,并不真正释放所占用的磁盘空间,这就导致InnoDB数据库文件不断增长。

如果在创建数据库的时候设置 innodb_file_per_table=1,这样 InnoDB 会对每个表创建一个数据文件,然后只需要运行OPTIMIZE TABLE 命令就可以释放所有已经删除的磁盘空间。
运行 OPTIMIZE TABLE 表名后,虽然最后会报 Table does not support optimize, doing recreate + analyze instead,但其实已经成功了。

那么如果没有设置这个参数,又想释放空间,彻底释放这些已经删除的数据,需要把数据库导出,删除InnoDB数据库文件,然后再倒入。

下面是基本的步骤:

  1. 使用mysqldump命令将InnoDB数据库导出
  2. 停止MySQL
  3. 删除所有InnoDB数据库文件和日志
  4. 启动MySQL并自动重建InnoDB数据库文件和日志文件
  5. 导入前面备份的数据库文件

具体DEMO:

# 备份数据库:
mysqldump -uroot -proot --quick --force --all-databases > mysqldump.sql
# 停止数据库
service mysqld stop
# 删除这些大文件
rm /usr/local/mysql/var/ibdata1
rm /usr/local/mysql/var/ib_logfile*
# 手动删除除Mysql之外所有数据库文件夹,然后启动数据库
service mysqld start
# 还原数据
mysql -uroot -proot < mysqldump.sql

由于项目以及服务器管理比较多,OSX上操作直接使用终端,无法如Windows Xshell一般记住密码,每次ssh的时候都要输入密码实在太浪费时间,今天有空一口气全部改为公钥/私钥认证,登录再也不用任何密码了。

实现步骤:

在你的自己的机器下面使用ssh-keygen命令来实现创建公钥,使用 ssh-keygen -t rsa 来创建密钥,程序会问你存放的目录,如果不需要修改,直接回车两次即可。

将你~/.ssh目录中的id_rsa.pub这个文件拷贝到你要登录的服务器的~/.ssh目录中,然后再运行以下命令来将公钥导入到~/.ssh/authorized_keys这个文件中

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

另外要注意请务必要将服务器上
~/.ssh权限设置为700
~/.ssh/authorized_keys 的权限设置为600
这是linux的安全要求,如果权限不对,自动登录将不会生效 完毕之后,退出服务器的登录,再使用ssh登录,你就会发现服务器不会再向你询问密码了.

当然也可以使用ssh-copy-id命令: ssh-copy-id [email protected]

ssh-copy-id -i 公钥文件路径 [email protected]
ssh-copy-id -p 服务器SSH服务端口 [email protected]

由于长期使用Composer进行包依赖管理,许多公用的类库都会放在GitHub上,在多个项目之间引用依赖,更新版本时,减少了很多成本,但还是存在一些问题:

  • 每次更新,本地需要跑过单元你测试,Commit并Push到Github,等待Composer更新后才生效;
  • 公司级的模块没有办法放在Github上;

官方方案

庆幸的事,Composer 就提供私有仓库的形式:

{
    "repositories": [
        {
            "type": "vcs",
            "url": "http://svn.example.org/projectA/",
            "trunk-path": "Trunk",
            "branches-path": "Branches",
            "tags-path": "Tags"
        }
    ]
}

具体参考:https://getcomposer.org/doc/05-repositories.md

由于公司内使用的是 Subverison,使用如上方式导入私有仓库,实际上是通过 svn checkout 的形式将指定的 tags 检出,这时候vendor目录下将会出现 .svn 目录,导致主项目无法提交,甚至在切换tags时出现无法检出的情况。

Composer项目issue中有许多人反映此问题,解决方案是通过 svn export 来代替 svn checkout,需要使用如下插件:
https://github.com/LinearSoft/composer-svn-export

新方案

该插件等于是需要自己搭建一个packaglist站点,其实仅需要一个packages.json文件,但是需要有效的https服务,否则将会出现如下错误:

➜  www.my-project.com composer update
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
Added SvnExport repo: NewPackage
Loading composer repositories with package information

                                                                                                                                             
  [Composer\Downloader\TransportException]                                                                                                   
  The "https://packagist.my-project.dev/packages.json" file could not be downloaded: SSL operation failed with code 1. OpenSSL Error messages:  
  error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed                                                          
  Failed to enable crypto                                                                                                                    
  failed to open stream: operation failed         

如果没有这样的https服务,可以选择在gist上建一个packages.json文件来达到效果:

{
    "packages": {
        "tc/analyse": {
            "0.9.2": {
                "name": "tc/analyse",
                "version": "0.9.2",
                "source": {
                    "type": "svn",
                    "url": "https://svn.tc.dev/library/analyse",
                    "reference": "/tags/0.9.2"
                }
            },
            "0.9.3": {
                "name": "tc/analyse",
                "version": "0.9.3",
                "source": {
                    "type": "svn",
                    "url": "https://svn.tc.dev/library/analyse",
                    "reference": "/tags/0.9.3"
                }
            },
            "dev-master": {
                "name": "tc/analyse",
                "version": "dev-master",
                "source": {
                    "type": "svn",
                    "url": "https://svn.tc.dev/library/analyse",
                    "reference": "/trunk"
                }
            }
        }
    }
}

主项目中composer.json填充上该Gist地址

{
    "require" : {
        "php" : ">=5.4.0",
        "tc/analyse" : "0.9.3",
        "linearsoft/composer-svn-export" : "^0.1.2"
    },
    "extra" : {
        "svn-export-repositories" : [
            {
                "name" : "TC",
                "type" : "composer",
                "url" : "https://gist.githubusercontent.com/lancerhe/379eeee89fd0db8c0c4ca19c6bddas2f/raw/915a720635949c9d4013746845b952f88af358db"
            }
        ]
    },
    "minimum-stability" : "stable"
}

运行 composer update

➜  www.my-project.com composer update
You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
Added SvnExport repo: TC
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Updating tapcash/analyse (0.9.2 => 0.9.3)
    Exporting /tags/0.9.3

Writing lock file
Generating autoload files

转眼已是2016年初三,今天正好回了一趟母校,最大的感触时间过得真快,已经毕业10年了,回想下来是该回头好好总结一下2015年。

先说说目标吧,当时14年结束的时候,在『百度无线91』给自己定了一个目标:研发架构师,回首再看确实目标定的有点高(估计一般都没有达成)!发现的同时在年中的时候我做了一个选择,从『百度无线91』离职,其主导原因是我希望能重新从『技术支持团队』回到『业务线团队』(毕竟我原来就是从业务线出来),当然也有其他一些复杂的原因,因此我现在就职于一家游戏公司(IGG)的广告业务团队,说起来还是干着广告这老本行。因此之前的计划:

  1. 保持读书计划,目标降低为12本,但是希望自己还是能超额完成。[未完成]
  2. 保持锻炼计划,身体好才是革命的本钱。[算是完成]
  3. 相比2014年的综合技能提升,其实专项技能的短板需要补足。[算是完成]
  4. 保持知识沉淀,合理利用Github。[完成]
  5. 给自己多一点话说的机会,学会引导技巧。[算是完成]
  6. 目标定位研发架构师,帮助团队整体水平提高。[未完成]

本文转载:aws ec2 硬盘 resize2fs

在申请 AWS EC2 时,按照向导,在选择存储的时候默认硬盘大小是 8G,这时候可以根据自己的需要输入一个合适的数字,例如100。完成向导并启动 EC2 instance 后登陆机器。使用命令:

df -hT

发现硬盘的大小不是自己的设定的值,而还是 8G,使用fdiskmkfs来分区和格式化后,还是无法增大其空间。反复折腾多次,包括重启机器,问题依旧,后来发现其实很简单,只需要使用一条命令resize2fs就可以搞定。

resize2fs /dev/xvde

注意:/dev/xvde 根据自己的实际情况可能会不一样。使用fdiskdf命令都可以获知具体的设备号。 如果执行上述命令收到 The filesystem is already 2096896 blocks long. Nothing to do! 的错误,那么需要先做如下操作:

<<1>> Look at the filesystem, it is 6G
<<2>> Look at the disk and the partition, the disk is 21.5 GB but the partition is 6 GB (6291456 blocks)
<<3>> Start fdisk for that disk (xvda, so not the partition xvda1)
<<4>> Switch to sector display.
<<5>> Print the partition(s), and remember the start sector (2048 in the example).
<<6>> Delete the partition.
<<7>> Create a new partition.
<<8>> Make it primary.
<<9>> First partition.
<<10>> Enter the old start sector, do NOT make any typo here!!! (2048 in the example) 
<<11>> Hit enter to accept the default (this is the remainder of the disk)
<<12>> Print the changes and make sure the start sector is ok, if not restart at <<6>>
<<13>> Make the partition bootable. do NOT forget this!!!
<<14>> Enter your partition number (1 in the example)
<<15>> Write the partition info back, this will end the fdisk session.
<<16>> Reboot the server, and wait for it to come up (this may take longer than usual).
<<17>> Verify the filesystem size.
<<18>> If the filesystem is not around 20Gb as expected, you can use this command.

# df -h  <<1>>

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      6.0G  2.0G  3.7G  35% / 
tmpfs            15G     0   15G   0% /dev/shm

# fdisk -l  <<2>>

Disk /dev/xvda: 21.5 GB, 21474836480 bytes
97 heads, 17 sectors/track, 25435 cylinders
Units = cylinders of 1649 * 512 = 844288 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003b587

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *           2        7632     6291456   83  Linux

# fdisk /dev/xvda  <<3>>

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): u  <<4>>
Changing display/entry units to sectors

Command (m for help): p  <<5>>

Disk /dev/xvda: 21.5 GB, 21474836480 bytes
97 heads, 17 sectors/track, 25435 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003b587

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *        2048    12584959     6291456   83  Linux

Command (m for help): d  <<6>>
Selected partition 1

Command (m for help): n  <<7>>
Command action
   e   extended
   p   primary partition (1-4)
p  <<8>>
Partition number (1-4): 1  <<9>>
First sector (17-41943039, default 17): 2048  <<10>>
Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): <<11>>
Using default value 41943039

Command (m for help): p <<12>>

Disk /dev/xvda: 21.5 GB, 21474836480 bytes
97 heads, 17 sectors/track, 25435 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003b587

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1            2048    41943039    20970496   83  Linux

Command (m for help): a  <<13>>
Partition number (1-4): 1  <<14>>


Command (m for help): w  <<15>>
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: ...
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

# reboot  <<16>>



# df -h  <<17>>
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       20G  2.0G   17G  11% / 
tmpfs            15G     0   15G   0% /dev/shm

# resize2fs /dev/xvda1  <<18>>
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/xvda1 is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 2
Performing an on-line resize of /dev/xvda1 to 5242624 (4k) blocks.
The filesystem on /dev/xvda1 is now 5242624 blocks long.

[email protected] [~]#  df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       20G  7.8G   11G  42% /
tmpfs           498M     0  498M   0% /dev/shm
/usr/tmpDSK     399M   11M  368M   3% /tmp

更多信息可以参考这里:http://serverfault.com/questions/414983/ec2-drive-not-ebs-volume-size