python基础和编程库-python黑洞网

数据类型
Number整数/浮点数
String字符串
Boolean布尔值True和False
空值None不是0
Tuple元组类型不必统一(1,‘abc’,0.4)
Dictionary key-value
List内置数据类型:列表[1,‘abc’,0.4],list是一种有序的集合，可以随时添加和删除其中的元素
变量
python是动态变量,不仅可以是数字，还可以是任意数据类型,是引用
与之对应的是静态变量,java是静态变量
字符串
字符串可以用’'或者""括起来表示。
如果一个字符串包含很多需要转义的字符，对每一个字符都进行转义会很麻烦。为了避免这种情况，我们可以在字符串前面加个前缀 r ，表示这是一个 raw 字符串，里面的字符就不需要转义了。

r'\(~_~)/ \(~_~)/'

在多行字符串前面添加r，把这个多行字符串也变成一个raw字符串

r'''Python is created by "Guido".
It is free and easy to learn.
Let's start learn Python in imooc!'''

语法

if age = 20
if age >= 18:
    print 'your age is', age
    print 'adult'
print 'END'

dict

d = {
    'Adam': 95,
    'Lisa': 85,
    'Bart': 59
}
print 'Adam:', d['Adam']
print 'Lisa:', d['Lisa']
print 'Bart:', d['Bart']

注释

单行注释用#
多行注释
“”"
“”"

编程库

1.time

import time
print(time.time())
time  = time.localtime( time.time() )
print(time)
print(time.tm_year)
"""
1563803665.310865
time.struct_time(tm_year=2019, tm_mon=7, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=25, tm_wday=0, tm_yday=203, tm_isdst=0)
2019
"""

2.Matplotlib

绘图工具包

3.Scikit-learn

封装了大量经典以及最新的机器学习模型

4.Pandas

针对于数据处理和分析的python工具包,实现了大量便于数据读写,清洗,填充及分析功能

4.1读取文件

import pandas as pd
# 两个数据类型：Series,  DataFrame

data_path = "C:/Users/admin/Desktop/111.csv"
# 读取文件
def read_file(data_path):
    datas = pd.read_csv(data_path, encoding="GBK")
    datas = datas.dropna()
    return datas
    
data = read_file(data_path)
print(data)

# 获取某一列文字   Python 字典(Dictionary) -->contents_agent 转化成一整段
contents_agent = data["asr_agent_raw"]
contents = contents_agent.values.tolist()

5.jieba

import jieba
stopwords_file = "D:/gitProject/smartlink-sqc/smartlink-sqc-wordle/python/dict/user_dict.txt"
def seg_word(contents):
    contents = contents.values.tolist()
    jieba.load_userdict(stopwords_file)
    segment = []
    for line in contents:
        try:
            segs = jieba.lcut(line)
            for seg in segs:
                if len(seg) > 1 and seg != '\r\n' and \
                        u'\u4e00' <= seg <= u'\u9fa5' or \
                        u'\u0041' <= seg <= u'\u005a' or \
                        u'\u0061' <= seg <= u'\u007a':
                    segment.append(seg)

        except:
            print(line)
            continue

    words_df = pd.DataFrame({'words': segment})
    stopwords = pd.read_csv(stopwords_file,
                            index_col=False,
                            quoting=3,
                            sep="\t",
                            names=['stopwords'],
                            encoding='utf-8')   # quoting=3全不引用
    words_df = words_df[~words_df.words.isin(stopwords.stopwords)]
    return words_df

# 进行分词
words_agent = seg_word(contents_agent)
print(words_agent)

6.NumPy & SciPy

NumPy最基础的编程库,提供一些高级的数学运算机制和高效的向量和矩阵运算功能
SciPy是子啊NumPy的基础上构建的,更为强大的科学计算包

import numpy as np
def word_freq(words_df):
    words_stat = words_df.groupby(by=['words'])['words'].agg({"count":np.size})
    words_stat = words_stat.reset_index().sort_values(by=["count"], ascending=False)
    return words_stat

words_stat_agent = word_freq(words_agent)
# 打印词频较高的前10 
print(words_stat_agent.head(10))